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The  findings,  recommendations,  and  opinions  expressed  in  this 
report  are  those  of  the  author  and  should  not  be  construed  as  an 
official  Department  of  the  Army  policy  or  position  unless  so 
designated  by  other  documentation. 


*  ** 


Much  hoopla  has  been  made  of  computers  in  recent  years.  Computer 
usage  is  both  fascinating  and  intimidating.  When  the 
microcomputer  revolution  began  in  the  mid-seventies,  some  of  us 
began  tinkering  with  this  new  technology  as  a  hobby.  As  these 
small  systems  became  more  powerful  and  easy  to  use,  they  started 
showing  up  in  offices,  first  as  cheap  word  processors,  then  as 
tools  to  do  more  challenging  and  analytical  work.  We,  in  the 
personnel  administration  business  became  more  aware  of  what 
computers  could  offer  in  helping  us  manage  the  tremendous  amount 
of  information  that  we  deal  with  on  a  daily  basis.  We  began  to 
question  the  versatility  and  functionality  of  standard  systems 
such  as  SCIPMIS,  CIVPBRCINS,  COEMIS,  etc.  These  "accounting" 
systems  are  harbored  in  the  depths  of  the  DMIS,  tended  by  minions 
who  loath  to  share  the  secrets  of  their  profession,  and  the  tools 
used  for  these  data  base  management  "wonders". 

We  began  to  see  that  computers  could,  and  should  do  more  than 
just  print  out  standard  reports  that  all  too  frequently  required 
manual  juggling  of  the  data  after  we  got  the  printout.  We  began 
to  demand  more  useful  and  powerful  systems,  systems  with  memory 
of  what  had  been  processed,  not  just  the  current  state  of  an 
employee's  record..  This  information  is  essential,  since  we  are 
monthly,  daily,  even  hourly  asked  to  provide  information  to 
others  about  events  such  as  "how  many  black  women  were  promoted 
during  the  last  quarter,  what  is  the  grade  break  out  of  those 
promoted,  and  how  does  that  compare  with  other  groups' 
promotions?".  Questions  such  as  these  require  manual  searching 
of  records,  since  our  standard  systems  can  only  tell  us  how  many 
black  women  we  have,  and  for  the  enterprising  sleuth  on  some  of 
our  systems,  how  many  were  promoted  during  the  last  quarter. 
Most  of  us  are  not  knowledgeable  enough  of  our  systems  to  know 
that  the  latter  part  just  might  be  available  with  a  lot  of  work 
on  the  part  of  a  programmer.  Most  of  us  use  the  manual  method  of 
answering  complex  questions  about  personnel  action  statistics. 
And  for  good  reason! 

Some  of  us  in  the  personnel  business  have  begun  to  see  that 
powerful  computer  systems  are  available  for  modest  cost,  and  are 
automating  our  offices.  We  all  seem  to  be  going  different 
directions,  but  at  least  some  of  us  are  automating  out  of  self 
defense.  There  is  a  cry  for  standardized  systems,  but  no  one 
dare  hold  back  the  movers  for  fear  of  being  asked  to  provide  a 
standard  solution  for  all.  And  have  it  next  week  since  we  all 
know  the  outcome  of  "real  soon  now"  by  experience. 

There  are  some  new  uses  of  computers  about  which  this  report  has 
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been  prepared.  These  uses  have  emerged  from  artificial 
intelligence  research  which  commenced  in  the  aid-sixties  and  are 
known  as  expert  systems.  I  began  to  get  interested  in  expert 
systems  in  aid-1983.  After  reading  all  1  could  lay  my  hands  on 
about  the  subject,  I  began  to  realize  that  expert  systems 
technology  could  be  applied  to  the  field  of  personnel 
administration.  I  subaitted  an  application  for  a  Secretary  of 
the  Army  Research  and  Study  Fellowship  (SARSF),  which  was 
subsequently  approved,  and  I  began  a  six  month  study  of  applying 
expert  systems  to  the  field  of  personnel  administration.  This 
report  is  the  results  of  ay  investigation. 

There  are  many  who  deserve  thanks  for  assistance  and  support  to 
me  in  carrying  out  this  project,  only  a  few  of  whom  I  have  room 
to  mention.  My  family  for  bearing  with  ay  being  away  for  six 
months;  Jim  Clark,  ay  XO  at  YPG  for  helping  to  get  the 
application  for  the  SARSF  off  the  installation;  Ray  Sumser',  for 
his  vision  in  recognizing  a  potential  idea  when  he  saw  it,  and 
putting  his  power  behind  approval  of  the  SARSF;  Conrad  Lacy  and 
others  on  the  DA  staff  for  their  help  with  information  and 
assistance  during  the  study;  Joe  Carlson  and  his  staff,  San 
Francisco  Field  Office  for  providing  the  environment  in  which  to 
conduct  the  study;  Severin  Johnson,  CPO,  Oakland  Army  Base,  for 
able  assistance  when  temporarily  detailed  to  this  project;  the 
many  personnelists  who  contributed  information  when  contacted; 
and  the  staff  of  the  Yuma  Proving  Ground  Civilian  Personnel 
Office  for  carrying  on  with  our  personnel  program  while  I  was 
away  on  this  project. 


There  is  nothing  more  powerful  than  an  idea  whose  time  has  come. 

-  Victor  Hugo  - 
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This  research  project  dealt  with  a  study  of  the  application  of 
expert  systems  technology  to  the  civilian  personnel 
administration  functions  in  the  Department  of  Army.'  Expert 
systems  technology  is  a  relatively  new  technique,  although  the 
idea  of  the  concept  has  been  around  at  least  since  the  first 

computer  was  designed  and  built.  In  science  fiction  literature, 

writers  have  used  their  lively  imaginations  to  spin  tails  about 
the  future  in  which  expert  systems  are  a  given,  and  aren’t  even 
called  by  a  name  at  all.  In  their  accepted  fashion,  computers 
and  robots  posses  "thinking"  capability  without  question.  In 
spite  of  the  computer’s  awesome  power  to  manipulate  data, 
attempts  to  program  it  to  simulate  human  brain  capability 

remained  allusive  for  many  years. 

In  the  1960’s  computer  scientists  began  to  place  a  heavy  emphasis 
on  artificial  intelligence  (AI)  usage  of  computers.  At 
universities  such  as  Carnegie-Mellon,  MIT,  Stanford,  UC  Berkeley, 
Rutgers  and  others,  and  in  company  research  departments  such  as 
Rand,  Xerox,  Bo^t,  Baranak  and  Newman,  SRI  International,  etc., 
AI  projects  were  undertaken.  These  projects  covered  such  areas 
as  linguistics,  robotics,  perception,  reasoning,  and  o'ther 
cognitive  science  areas.  One  general  area  that  began  to  show 
considerable  promise  in  terms  of  practical  applications  was 

expert  systems. 

^Expert  systems  are  computer  programs  that  tackle  complex  problems 
which  require  a  specialist  or  expert  to  solve,  and  provides 
solutions  to  the  problems  using  a  computer  model  of  the  human 
expert’s  reasoning  techniques.  The  solution  to  the  problem 
provided  by  the  expert  system  is  the  same  one  that  the  human 
expert  would  have  reached.  In  this  section  of  the  report,  I  will 
review  several  expert  systems  that  have  been  developed.  This 
will  give  the  reader  an  understanding  of  what  expert  systems  are, 
and  will  give  credence  to  the  idea  that  they  actually  work.  I 
will  not  attempt  to  go  into  detail  as  to  how  they  do  what  they 
do.  Such  a  subject  is  beyond  the  scope  of  this  research  project, 
and  in  many  ways  beyond  my  capability  to  explain.  The 
technically  inquisitive  reader  may  refer  to  Appendix  A  as  a 
starting  point  for  further  reading  in  this  area.  The  references 
also  contain  their  own  references  for  further,  in-depth  reading. 

MYCIN.  This  expert  system  was  developed  in  the  mid-1970’s,  and 
was  designed  to  diagnose  and  prescribe  treatment  for  infectious 
blood  diseases.  When  used  by  a  physician,  the  program  prompts 
the  doctor  for  information  about  the  patient,  results  of  tests 
conducted  on  specimens  from  the  patient,  and  observations  made  of 
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the  patient  by  the  physician.  As  the  questioning  by  MYCIN 
progresses  with  responses  from  the  doctor,  the  program  uses 
expert  knowledge  and  reasoning  techniques  to  infer  a  diagnosis. 
After  a  diagnosis  is  rendered,  MYCIN  proceeds  to  query  for 
additional  information  so  that  a  therapy  recommendation  can  be 
made.  MYCIN  will  then  prescribe  the  drugs  to  be  used  to  combat 
the  diagnosed  infection  in  the  patient.  This  expert  system  has 
been  extensively  tested  by  medical  experts.  Tests  have  shown  it 
to  be  as  good  as,  or  superior  to  physicians  in  diagnosing  and 
prescribing  treatment  in  the  specialized  domain  of  medical 
knowledge  that  it  represents.  One  of  the  major  results  of  the 
MYCIN  project  was  a  by-product  called  EMYCIN.  This  program  is  a 
framework  (programming  environment),  or  tool  which  is  generalized 
in  nature  and  application  independent.  It  can,  and  has  been  used 
to  develop  other  expert  systems.  One  could  view  it  as  MYCIN 
without  its  specific  knowledge  of  bacteremia.  The  EMYCIN  system 
has  been  used  to  produce  another  well-known  medical  application 
called  PUFF,  which  is  concerned  with  pulmonary  functions. 

There  have  been  some  other  notable  expert  systems  developed  for 
medical  related  knowledge  domains  such  as  CAOUCEUS  and  INTERNIST 
which  both  work  in  the  area  of  internal  medicine  diagnosis  and 
treatment,  and  CASNET  which  is  specific  for  glaucoma  diagnosis 
and  therapy. 

PROSPBCTOR.  This  system  works  in  the  knowledge  domain  of  mineral 
exploration.  Specific  types  of  minerals  for  which  models  have 
been  developed  for  PROSPBCTOR  are  sulfide,  carbonate  -lead/zinc, 
porphyry  copper,  nickel  sulfide,  and  uranium.  The  system  can  be 
used  by  geologists  and  others  who  know  the  geological  terms  used 
by  the  system  to  query  for  information  from  the  user.  Much  of 
the  query  process  uses  requests  for  confidence  measures  from  the 
user.  Thus  a  question  such  as  "To  what  degree  do  you  believe 
that  the  target  area  lies  in  a  greenstone  belt?"  The  user  must 
supply  a  number  from  0  to  5  representing  his  confidence  that  this 
is  true.  Obviously,  one  has  to  know  what  a  greenstone  belt  is, 
and  be  able  to  supply  some  assessment  of  the  area  under 
consideration  as  to  whether  it  is  one  or  not.  PROSPECTOR  will 
also  accept  volunteered  information  from  the  user.  Thus  it 
offers  another  dimension  to  a  session  other  than  only  querying 
for  responses  from  the  user.  This  is  termed  a  mixed  initiative 
mode  and  gives  the  user  an  additional  means  of  control  over  a 
session  with  the  expert  system.  At  the  conclusion  of  the  session 
with  PROSPBCTOR,  the  system  displays  an  index  showing  the 
favorableness  of  the  mineral  being  present  in  the  target  area. 
PROSPECTOR  has  been  credited  with  finding  a  molybdenum  deposit 
whose  ultimate  value  will  probably  exceed  $100  million. 

Rl.  This  is  an  expert  system  presently  in  real  world  use  by 
Digital  Equipment  Corporation  which  configures  VAX  computers. 
VAX  computers,  as  well  as  most  mini  and  main  frame  systems,  are 
put  together  from  a  wide  range  of  sub-assemblies.  These  sub- 
assemblies  can  be  such  things  as  different  back-planes  (a  kind  of 
socket  board  that  accepts  plug-in  boards),  power  supplies,  memory 
boards,  central  processor  boards,  boards  to  handle  input  and 
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output  to  other  devices  such  as  disk  drives  and  terainals,  and 
other  such  things.  A  VAX  computer  can  be  configured  in  a 
multitude  of  different  ways.  When  a  customer  purchases  a  VAX 
computer,  it  is  configured  to  meet  the  customer's  specific  needs. 
Given  the  number  of  parts  and  sub-assemblies  involved,  it  is  not 
readily  apparent,  without  considerable  expertise,  to  insure  all 
of  the  necessary  pieces  have  been  ordered,  will  work  together, 
and  are  shipped  so  that  the  product  can  be  assembled  to  meet  the 
needs  of  the  customer.  DBC  used  to  use  highly  trained 
technicians  to  review  purchase  orders  and  shipping  documents  to 
insure  all  parts  were  included,  and  the  system  was  configured 
properly.  This  job  is  now  performed  by  Rl.  R1  also  produces  a 
set  of  diagrams  depicting  the  spatial  relationships  between  the 
components. 

DBNDRAL.  This  system's  expertise  lies  in  the  knowledge  domain  of 
chemicals.  DBNDRAL  accepts  queries  from  users  to  apply  expert 
decisions  about  the  plausible  structures  of  an  unknown  compound. 
The  system  uses  data  supplied  by  the  user  which  is  mass 
spectrographic,  nuclear  magnetic  resonance  and  other  chemical 
experiment  data  and  applies  a  large  knowledge  base,  plus  an 
efficient  inferencing  technique  to  draw  conclusions.  DBNDRAL, 
which  is  nearing  two  decades  of  existence,  was  one  of  the 
earliest  expert  systems  developed  for  a  practical  problem.  It  is 
currently  used  by  hundreds  of  international  users  daily  for 
chemical  structures  analysis. 

While  there  are  a  number  of  other  expert  systems  in  existence, 
and  more  being  developed,  the  above  examples  will  give  some 
indication  of  what  expert  systems  are  capable  of  doing.  Recently 
companies  have  been  exploring  new  uses  such  as  systems  that  do 
loan  analysis  for  lending  institutions.  Such  systems  may 
eventually  replace  experts  known  as  loan  officers.  Expert 
systems  to  help  fly  the  space  shuttle,  diagnose  problems  on 
satellites,  assist  mechanics  repairing  diesel-electric 
locomotives,  and  a  wide  variety  of  other  knowledge  domains  are 
either  under  development  or  already  exist. 
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As  mentioned  earlier,  it  is  not  within  the  scope  of  this  project 
to  thoroughly  explain  to  the  casual  reader  how  expert  systeas  do 
what  they  do.  Soae  explanation  of  how  they  operate  is  necessary 
in  order  to  report  on  potential  tools  that  Bight  be  used  by  a 
knowledge  engineer  in  building  a  personnel  administration  expert 
systea. 

As  opposed  to  conventional  procedural  programming,  in  which 
specific  procedures  are  laid  out  in  the  program  to  control  its 
flow,  AI  programming  uses  descriptive  techniques  in  which 
solutions  to  probleas  are  described,  not  prescribed.^  Lest  this 
all  sound  like  a  bunch  of  high-tech  gobbledygook ,  perhaps  an 
example  will  help  to  clarify  the  differences. 

Suppose  a  program  is  being  written  to  solve  the  problem  of 
finding  a  naae  in  a  data  base  of  several  hundred  names.  The 
programmer,  using  a  conventional  procedural  language  would  write 
a  procedure  that  would  ask  the  user  for  the  name  being  sought. 
When  the  name  is  input,  the  program  has  a  specific  procedure  that 
says,  assign  the  input  name  to  a  variable,  compare  this  variable 
with  each  naae  variable  in  the  data  base,  if  a  match  is  found, 
select  the  entire  record  associated  with  that  name  and  print  it 
out  on  the  screen,  if  a  match  is  not  found  print  .the  message 
"Name  not  found”  on  the  screen.  In  an  AI  programming  language 
such  as  PROLOG,  the  programmer  describes  the  solution,  and  the 
programming  language  solves  the  problem.  For  example,  on  the 
same  problem,  the  programaer  would  write  a  description  of  the 
solution  that  says  the  "solution  looks  like  this”,  and  the 
programming  language  would  solve  the  problem. 

Expert  systeas  have  been  aodeled  on  several  different  kinds  of 
inferencing  techniques.  Soae  of  these  techniques  will  be 
discussed. 

Production  rules  are  necessary  for  most  expert  systems  operation. 
A  production  rule  takes  the  form  of  IF  -  THEN  construct.  The  IF 
portion  of  the  rule  is  tested  by  the  program  based  on  facts  known 
to  the  system.  If  this  test  is  found  to  be  true,  the  THEN 
portion  of  the  rule  is  asserted.  When  this  occurs,  it  is  known 
as  a  rule  firing.  An  example  of  an  IF  -  THEN  rule  might  look 
like  this  (in  English): 

IF  the  employee  is  required  to  have  contact  with  Members  of 
Congress,  AND 

IF  the  contacts  are  for  the  purpose  of  presenting  major  portions 
of  the  agency’s  budget,  AND 
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IF  these  contacts  occur  at  regular  intervals 


THEN  assign  a  value  of  110  points  for  CONTACTS  with  a  certainty 
factor  of  .85 

In  this  example  (which  might  be  a  rule  in  classifying  a  Budget 
Analyst  position),  the  rule  is  fired  if,  and  only  if  all  three  IF 
clauses  are  true.  Other  rules  may  have  lead  up  to  this  rule,  for 
example,  a  production  rule  might  have  been  fired  to  determine 
what  is  meant  by  "major  portions  of  an  agency’s  budget".  As  can 
be  seen  in  this  example,  a  certainty  factor  can  be  assigned  to  a 
rule.  This  is  important  to  many  rule  based  expert  systems,  since 
it  is  frequently  desirable  to  combine  several  certainty  factors 
before  actually  making  a  decision. 

Another  technique  that  is  used  in  expert  systems  is  a  method  of 
representing  knowledge  as  objects.  These  objects  are  usually 
called  frames,  and  have  unique  characteristics.  A  frame  can  have 
many  slots  as  they  are  called  which  serve  to  describe  and  link 
the  frame  to  other  frames.  An  example  of  a  frame  is: 

POSITION,  which  may  have  slots  which  describe  it  such  as 
Title,  Series,  Grade,  Knowledge  Required,  Supervisory  Controls, 
Incumbent,  etc.  A  frame  slot  can  be  filled  with  facts,  for 
example  11  for  grade,  and  can  be  linked  to  other  frames  by  common 
slots,  for  example  an  incumbent’s  name  could  occupy  a  slot  in 
another  frame  called  SUPERVISOR.  A  slot  can  also  be  occupied  by 
a  procedure  such  as  a  procedure  to  calculate  age  for  the  slot 
called  AGE. 

Many  complex  and  powerful  relationships  can  be  represented  by  a 
frame  based  system.  Many  expert  systems  utilize  both  frames  and 
production  rules  in  combination,  which  provide  a  great  deal  of 
reasoning  ability. 

The  ability  of  expert  systems  to  draw  inferences  and  make 
decisions  is  accomplished  in  the  inference  engine  of  the  system. 
Rules,  reasoning  strategies  and  control  mechanisms  constitute  the 
basis  of  the  inference  engine.  For  example,  a  control  mechanism 
is  necessary  to  insure  the  expert  system  knows  how  to  start,  and 
proceed  through  a  decision  making  process,  and  to  resolve 
conflicts  when  they  arise.  A  reasoning  strategy  that  uses 
forward  chaining,  or  backward  chaining,  or  a  combination  of  both 
can  be  utilized.  Forward  chaining  involves  applying  facts  to 
rules,  and  as  rules  fire,  using  this  new  knowledge  to  fire  other 
rules  until  the  system  gains  enough  information  to  draw  a 
conclusion.  Backward  chaining  occurs  when  the  system  chooses  a 
hypothesis  to  prove,  and  then  chains  backward  to  gain  facts  that 
would  support  this  hypothesis  until  the  hypothesis  is  either 
proven  or  disproven.  Control  mechanisms  are  also  required  to 
optimize  the  search  by  the  system  to  make  inferences  about  the 
facts  present.  For  example  a  breadth  search  technique  causes  the 
system  to  sweep  across  all  premises  in  a  rule  before  digging  for 
greater  detail  about  subgoals.  A  depth  search  digs  into  details 
about  a  particular  subgoal  before  going  on  to  other  subgoals.  As 
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a  way  of  illustration,  a  breadth  first  search  in  the  area  of  job 
classification  may  cause  the  questioning  to  jump  from  a  question 
about  knowledge  to  one  about  personal  contacts,  and  then  to 
supervisory  controls,  and  back  to  get  more  information  about 
knowledge.  A  depth  first  search  would  result  in  all  questions 
about  knowledge  being  gathered  first  before  moving  on  to 
supervisory  controls. 

Many  of  the  references  in  Appendix  A  cover  these  techniques  in 
considerable  detail,  and  the  inquisitive  reader  interested  in 
building  a  better  foundation  in  techniques  of  knowledge 
representation  and  reasoning  strategies  in  expert  systems  is 
encouraged  to  read  these  materials. 

It  would  appear  to  me  that  the  civilian  personnel  administration 
problem  is  subject  to  solution  by  use  of  production  rules.  For 
the  most  part,  solutions  to  any  problem  in  our  business  revolves 
around  the  application  of  rules  to  the  problem  at  hand.  For 
example,  the  qualifications  determination  on  an  employee  utilizes 
rules  based  on  the  kinds,  amount  and  quality  of  experience  and 
training.  Staffing  specialists  use  published  guides,  some  of 
which  are  very  specific  and  prescriptive,  and  rules  of  thumb 
gained  through  experience  in  making  qualifications 
determinations.  These  rules  can  be  embodied  in  an  expert  system 
using  production  rule  techniques  of  AI  programming-.  Frame  based 
techniques  also  offer  advantages  in  linking  a  variety  of  facts 
about  related  objects  (people,  positions,  etc.)  into  an 
integrated  knowledge  base.  Utilization  of  powerful  software 
development  tools  that  have  many  of  these  features  available  will 
allow,  for  the  development  of  an  expert  system  capable  of  making 
expert  decisions  about  personnel  matters. 
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The  actual  development  of  an  expert  system  has,  in  the  past,  been 
done  by  a  very  highly  specialized  individual  known  as  a  knowledge 
engineer.  Knowledge  engineers  possesses  skills  and  knowledges  of 
expert  systems  structures,  i.e.  the  various  inferencing 
techniques  that  have  been  successfully  utilized.  They  also  are 
skilled  at  programming  (although  few  actually  write  the  code  for 
computer  programs),  and  have  highly  developed  abilities  in 
interviewing  experts  in  other  knowledge  domains,  and  narrowing 
the  knowledge  base  down  to  the  specific  rules  the  experts  use  in 
making  decisions  and  judgments.  The  knowledge  engineer 
constructs  an  expert  system  either  through  the  use  of  an  existing 
expert  system  building  tool,  or  through  directing  the  programming 
efforts  of  programmers  who  write  code  in  a  high  level  language 
(usually  LISP  or  PROLOG).  Some  of  the  tools  available  for 
building  expert  systems  are:  EMYCIN  (which  was  mentioned  above) , 
ROSIE,  KAS,  BXPERT ,  0PS5,  RLL,  HEARSAY- III,  AGE,  KEE  and  ART,  to 
name  a  few. 

These  tools  run  on  mini  or  main-frame  computers.  Most  of  them 
are  products  of  large  scale  expert  system  development  programs. 
EMYCIN  resulted  from  the  MYCIN  project,  HEARSAY-III  from  work  on 
speech  understanding  systems,  and  KAS  from  the  work  done  on 
PROSPECTOR.  Bach  of  these  tools  are  proficient  in  specific  kinds 
of  expert  systems  engineering.  Some  attempts  have  been  made  to 
evaluate  the  capabilities  of  them  in  a  comparative  mode,  having 
an  identical  problem  being  solved  using  each  system  (see  Hayes- 
Roth,  Waterman  and  Douglas). 

There  are  several  stages  to  the  development  of  an  expert  system. 
Various  publications  have  laid  out  the  stages,  and  many 
recommendations  for  development  are  available.  The  following 
stages  are  a  synthesis  of  these  recommendations  as  I  believe  they 
apply  in  this  particular  application. 

Stage  1.  General  Assessment.  This  involves  assessing  the 
feasibility  of  applying  the  technology  to  the  domain  of  knowledge 
under  consideration.  It  includes  an  analysis  of  potential  cost- 
benefit  ratios,  what  areas  of  the  knowledge  domain  could  benefit 
by  expert  systems  usage,  what  policy  or  other  issues  need  to  be 
addressed,  what  resources  are  needed,  etc.  This  report  serves  as 
the  Stage  1  process. 

Stage  2.  Design  and  Build  Prototype.  This  stage  is  an  R&D 
effort  to  actually  build  a  prototype  system  for  the  knowledge 
domain  selected.  The  tool  or  language  to  be  used  is  selected,  a 
strategy  for  attacking  the  problem  is  formulated,  and  work  begins 


on  developing  the  prototype.  At  this  stage,  several  false  starts 
■ay  be  experienced,  but  the  purpose  is  to  get  a  prototype 
operational  for  further  test  and  development. 

Stage  3.  Development.  Knowledge  is  added  to  the  system,  and 
experts  are  used  to  further  expand  on  the  ability  of  the 
prototype  to  make  accurate  decisions.  During  this  stage,  the 
goal  is  to  insure  the  prototype  is  ready  to  be  tested  in  an 
actual  setting.  The  scope  of  the  prototype  may  not  encompass  the 
entire  knowledge  domain,  but  the  scope  that  has  been  selected 
must  be  thoroughly  and  completely  developed. 

Stage  4.  Test  and  Validation.  During  this  stage,  the  prototype 
is  put  to  actual  use.  Its  performance  is  closely  monitored,  and 
ideally  redundant  work  is  performed  by  human  experts  to  check  the 
validity  of  the  system.  During  this  stage,  further  development 
■ay  be  indicated  by  the  results  achieved  in  this  beta  test  mode. 

Stage  5.  Implementation  Planning.  This  stage  will  identify  the 
means,  methods  and  time  schedules  for  putting  the  prototype  on 
line  for  productive  use. 

Stage  6.  Implementation.  The  system  is  designated  a  standard 
system  and  distributed  for  mandatory  use  by  all  users  who  are 
affected  by  its  knowledge  domain. 

Stage  7.  Maintenance  and  Bxtension.  This  stage  is  continuing  in 
nature.  All  expert  systems  require  maintenance  to  keep  them  up- 

to-date  with  new  knowledges.  If  the  prototype  had  limited  scope 
‘in  the  applicable  knowledge  domain,  then  extension  of  the  system 
will  be  required  until  the  entire  knowledge  domain  can  be 
provided,  by  the  system. 

Stages  5,  6,  and  7  are  stages  that  will  occur  assuming  a 

successful  prototype  is  developed.  They  are  routinely  not 
addressed  to  any  degree  until  the  prototype  has  been  developed. 
They  are  presented  here  only  to  reflect  the  experience  of  real 
systems  that  have  been  developed  such  as  HI.  They  can  be 
anticipated,  but  not  well  planned  for  early  in  the  assessment 
process . 
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A  hands-on  test  of  applying  expert  systems  technology  to  a 
civilian  personnel  function  was  undertaken  during  the  course  of 
this  project.  This  exaaination  served  two  purposes.  It  allowed 
■e  to  try  actually  developing  an  expert  system,  as  linited  as  it 
was,  and  it  produced  a  functioning  deaonstration  module  to  allow 
others  to  see  how  expert  systems  technology  night  be  implemented 
in  the  civilian  personnel  administration  area. 

A  software  package  called  Expert  Base  was  used  for  development  of 
the  demo  module.  This  program  is  intended  to  be  an  expert  system 
development  tool  for  non-programmers  to  use  in  developing  expert 
systems.  It  had  its  limitations,  but  served  the  purpose  of 
allowing  me  to  prepare  a  small  demo  module.  The  functional  area 
chosen  for  the  demo  module  was  job  classification,  primarily 
because  it  appeared  to  offer  the  most  straight  forward 
application,  and  also  because  I  have  a  background  in  position 
classification.  Additionally,  there  were  other  personnel 
available  in  the  San  Francisco  Field  Office  to  examine  the 
operation  of  the  module  and  offer  suggestions  on  how  to  improve 
.its  performance  in  arriving  at  correct  classifications  of  jobs. 

In  order  to  keep  the  expert  system  on  familiar  ground,  I 
approached  the  design  of  the  expert  system  to  mimic,  as  closely 
as  possible,  the  OPN  classification  system.  This  gave  me  some 
insight  into  the  problems  that  would  be  faced  by  an  expert  system 
developer  if  a  full  scale  project  were  undertaken  to  mimic  our 
current  system,  and  it  would  result  in  a  demo  module  that  would 
be  recognizable  by  others  who  would  see  it. 

I  first  attempted  to  develop  a  module  to  classify  a  secretarial 
job.  This  attempt  was  abandoned  after  working  with  it  for  a 
couple  of  days.  It  proved  to  be  extremely  complex  to  try  to 
cause  the  system  to  mimic  OPM  classification  guidance,  and  Expert 
Ease  did  not  appear  to  have  the  capability  to  encompass  the 
problem  without  a  very  convoluted  mass  of  files.  This  would 
cause  the  demo  module  to  run  very  slowly  just  to  classify  one 
secretarial  job  since  it  would  have  to  call  in  a  number  of  files 
to  fully  examine  all  the  factors  involved  in  the  classification 
process.  As  discussed  elsewhere  in  this  report,  it  may  not  be  a 
wise  choice  to  try  to  develop  expert  systems  for  the  civilian 
personnel  administration  functions  that  exactly  mimic  current 
systems.  Although  I  believe  they  can  be  so  developed,  perhaps 
the  investment  in  time  it  would  take  to  cause  this  to  occur  would 
not  be  a  cost  effective  utilization  of  expert  systems  technology. 
This  needs  to  be  more  fully  explored  with  better  software  tools 


than  I  had  available. 

1  turned  to  the  wage  grade  area  to  work  up  the  demo  module.  It 
was  a  relatively  easy  task  to  prepare  the  module.  In  order  to 
have  some  variety  in  it,  I  chose  three  broad  occupational  groups; 
equipment  maintenance  and  repair,  operation  of  equipment,  and 
food  service  and  preparation.  Within  these  three  broad 
categories,  I  chose  the  specific  occupations  of;  Automotive 
Mechanic,  Mobile  Equipment  Servicer,  Air  Conditioning  Equipment 
Mechanic,  Fork  Lift  Operator,  Crane  Operator,  Motor  Vehicle 
Operator,  Cook,  and  Food  Service  Worker. 
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Figure  1:  Flow  chart  for  Wage  Grade  demo  module  of 
expert  system. 


The  deao  Module  is  composed  of  eleven  files.  The  main  file, 
called  WAGE. GRAD  branches  to  six  other  files,  two  of  which  branch 
to  two  other  files  each.  The  flow  chart  for  this  file  structure 
is  depicted  in  Figure  1. 

A  forward  chaining  technique  is  utilized  in  the  module.  This 
technique  uses  decisions  reached  in  one  file  to  cause  forward 
chaining  to  the  appropriate  file  for  further  decisions.  In 
practice,  this  technique  could  have  been  used  to  classify  the 
secretary  job. 

The  user  interface  of  the  demo  module  employs  menu  selection 
choices  and  some  numerical  input  responses  requested  of  the  user 
by  the  systea.  This  interface  choice  was  driven  by  the  Expert 
Ease  software.  More  powerful  systems  development  tools,  such  as 
HEARSAY-II  and  III,  and  AGE  provide  some  natural  language 
interface  capability,  thus  a  more  free  flowing  interaction 
between  the  system  and  the  user  can  be  designed. 

The  deao  aodule  begins  by  querying  the  user  for  the  main  category 
in  which  the  job  falls.  It  then  asks  pertinent  questions,  if 
necessary,  to  further  clarify  for  itself  which  classification 
file  it  will  use  to  classify  the  job.  The  demo  module  eventually 
locates  the  appropriate  file  to  be  used  to  classify  the  job.  A 
series  of  questions  are  then  presented  to  the  user  to  elicit 
information  that  will  be  used  in  the  actual  job  classification 
process  by  the  expert  system  demo  aodule. 

Different  modeling  schemes  were  used  in  the  development,  of  these 
files  to  be  used, in  the  classification  process.  For  example,  a 
rather  simple  model  was  used  for  Air  Conditioning  Mechanic  which 
is  dependent  on  asking  the  user  about  the  degree  of  supervision 
provided  to  the  employee  of. the  job  being  classified,  up  to  ten 
questions  about  knowledges  required  of  the  worker,  and  a  question 
about  whether  air  conditioning  equipment  being  repaired  is 
directly  involved  in  experiments  or  tests  of  other  things.  This 
model  will  result  in  a  grade  of  WG-11  if  little  or  no  technical 
supervision  is  exercised  over  the  employee,  work  does  involve  a 
"testing”  environment,  and  nine  or  ten  of  the  knowledges  are 
specified  as  being  required.  If  only  eight  of  the  knowledge 
questions  are  answered  affirmatively,  and  the  other  elements  are 
the  same,  a  grade  of  WG-10  results.  If  work  is  not  done  on 
"test"  A/C  equipment,  and  eight  or  more  knowledge  questions  are 
answered  in  the  affirmative,  a  WG-10  results.  If  any  number  of 
knowledge  questions  less  than  eight  are  answered  affirmatively, 
or  if  supervision  is  closer  than  a  certain  level,  then  a  grade  of 
WG-8  is  assigned.  This  model  was  chosen  to  partially  guard 
against  those  who  would  try  and  cheat  the  system.  It  assumes  any 
journeyman  A/C  mechanic  would  need  to  possess  most,  if  not  all  of 
the  knowledges  represented  by  the  knowledge  questions.  It 
prevents  a  WG-11  classification  unless  the  A/C  mechanic  is 
actually  working  on  equipment  directly  used  in  experiments  and 
tests.  The  model  is  designed  to  be  on  the  conservative  side  in 
allocating  the  job. 
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The  aodel  as  described  above  is  not  known  to  the  user,  nor  are 
any  of  the  Models  in  the  deMO  Module.  Bxpert  Ease  does  not 
provide  a  facility  so  that  the  user  can  ask  the  system  why  a 
particular  question  is  being  asked.  Most  expert  systems  in  use 
today  do  provide  a  Means  for  the  user  to  ask  the  system  why  it  is 
seeking  any  particular  piece  of  information.  The  system  will 
respond  and  advise  the  user  of  the  rule  it  is  trying  to  use,  or 
give  other  indication  back  to  the  user  what  the  information  being 
sought  will  be  used  for. 

While  the  A/C  Mechanic  model  is  relatively  simple  in  concept, 
Expert  Base  had  a  dickens  of  a  time  dealing  with  it  in  inducing 
the  rules  to  be  used  in  making  a  decision.  After  setting  up  the 
attributes  and  values,  I  had  to  develop  188  examples 
(combinations  of  responses  to  attributes)  for  the  system  to  use 
in  the  induction  process.  A  rule  (decision  tree)  of  373  nodes 
resulted,  and  it  took  the  system  15  minutes  to  Make  the 
induction.  Normally,  a  30  node  rule  is  about  all  Expert  Ease  is 
equipped  to  handle.  From  the  program's  point  of  view,  this  is  a 
very  complex  model. 

Other  models  used  were  somewhat  more  complex  from  a  personnel 
viewpoint,  but  relatively  simple  from  the  program's  point  of 
view.  For  example,  the  fork  lift  operator  was  initially  modeled 
on  the  concept  that  if  the  operator  were  required  to  operate  a 
fork  lift  of  10,000  pounds  or  more,  a  grade  of  WG-06  resulted, 
and  if  the  fork  lift  had  a  capacity  of  less  than  10,000  pounds,  a 
WG-5  classification  was  made.  After  discussing  this  model  with 
Mr.  Bill  Dittmar  in  the  Field  Office  (an  expert  classifier),  the 
model  was  changed  to  get  qt  the  subtle  differences  in  outside 
work  with  a  fork  lift  vs.  inside  work.  This  expert  consultant 
input  was  used  to  modify  the  model  so  that  the  system  will 
produce  a  WG-5  even  if  the  fork  lift  is  over  10,000  pounds, 
providing  certain  criteria  are  not  present,  and  could  even  result 
in  a  WG-6  for  an  operator  with  a  small  fork  lift  under  certain 
conditions.  Although  further  refinement  could  be  made  to  the 
model  to  more  accurately  carve  out  the  criteria  necessary  for  a 
classification  that  is  not  necessarily  inextricably  tied  to  the 
capacity  of  the  fork  lift,  the  refinements  made  do  point  out  the 
flexibility  of  designing  expert  systems  that  follow  the  logical 
thought  processes  of  experts,  and  give  expert  answers. 

Some  conclusions  can  be  drawn  as  a  result  of  using  Expert  Ease  to 
develop  the  demonstration  module: 

1.  A  much  more  powerful  expert  system  tool  will  be  required 
to  attempt  to  create  any  usable  expert  systems  for  the  personnel 
administration  functions.  This  was  not  a  surprising  conclusion. 
Our  business  is  entirely  too  complex  for  any  but  the  most 
powerful  applications  software.  Using  a  language,  such  as  LISP 
or  PROLOG  is  a  viable  option  in  expert  systems  development  for 
personnel  administration.  Some  of  the  tools  on  the  market  such 
as  BMYCIN,  ROSIE,  HEARSAY-III,  and  others  may  posses  the  power  to 
handle  the  problem.  The  choice  of  using  a  tool,  a  language,  or 
both  will  need  to  be  decided  by  the  actual  developer  of  the 
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systea,  should  a  system  be  approved  for  developaent. 

2.  A  aajor  decision  will  need  to  be  Bade  whether  or  not  an 
expert  systea  for  personnel  administration  should  exactly  mimic 
current  systems,  or  if  new  policies  and  changes  in  systems  will 
be  implemented  as  the  systea  is  developed.  This  is  a  very 
critical  issue  and  requires  a  policy  decision.  It  became  obvious 
to  me  as  the  module  was  being  developed,  that  the  expert  used  in 
shaping  the  decision  processes  the  system  will  imitate,  is  the 
crux  of  the  entire  system.  For  example,  one  could  find  little 
reason  to  make  any  further  distinctions  in  classifying  a  fork 
lift  operator  job  than  the  capacity  of  the  fork  lift.  One  could 
further  refine  the  system  to  get  at  more  subtle  differences  in 
work  performed,  as  I  did  with  the  demo.  The  final  demo  could  be 
even  further  refined  to  account  for  a  multitude  of  minute  nuances 
pertinent  to  a  fork  lift  operator  job  ad  infinitum.  The  choice 
of  just  how  detailed  one  wants  to  make  the  system,  and  whether  or 
not  new  policies  are  instituted  as  a  system  is  developed  is  an 
important  one.  Due  to  a  need  to  have  some  basis  of  comparison 
between  our  current  system  and  one  operated  under  an  expert 
systea,  I  believe  such  a  systea  should  mimic  our  current  system. 
This  would  also  make  it  easier  to  find  "experts"  for  the 
developaent  process.  However,  any  such  system  would  mimic  THAT 
expert  or,  THOSE  experts  used  in  the  development. 

3.  This  brings  me  to  experts.  It  would  be  clearly  a 
disaster  to  attempt  to  build  an  expert  systea  using  the  committee 
approach  to  "expertise".  A  knowledge  engineer  attempting  to. deal 
with  a  group  of  "expmrts"  in  developing  a  system  would  never 
complete  .  the  task.  This  is  not  to  say  that  additional  experts 
should  not  be  used  to  test  the  system,  and  make  adjustments.' 
This  is  a  normal  practice  in  expert  systems  development.  This  is 
also  not  to  say,  that  different  experts  should  not  be  used  in 
different  areas  of  expertise,  they  most  certainly  should  be. 
However,  during  the  initial  developaent  stages,  the  knowledge 
engineer  would  need  to  have  available  the  representative  that 
would  serve  as  the  model  whose  knowledge  and  expertise  will  be 
used  for  modeling  the  system’s  expertise. 

4.  It  is  possible  to  build  in  many  checks  and  balances  to 

the  systea.  For  example,  in  the  area  of  classification,  a  fear 

might  be  that  the  supervisor  could  manipulate  a  particular  grade 
out  of  the  system  if  he  or  she  were  knowledgeable  of  the 

standards  or  the  modeling  that  went  into  the  building  of  the 
system.  This  is  not  too  unlike  our  current  system,  however,  we 
enjoy  the  added  benefit  of  a  classifier  on  site  to  demand  proof 
of  claims  about  jobs.  It  must  first  be  decided  whether  or  not 
this  kind  of  checking  is  important.  And  this  is  a  very 

significant  decision.  If  it  were  decided  that  some  method  of 
verification  needed  to  be  provided  for  in  the  systea  to  guard 
against  unscrupulous  supervisors,  then  there  are  ways  to  do  this. 
For  example,  in  the  area  of  classification,  the  employee  could  be 
required  to  interact  with  the  system  on  a  periodic  basis  to 
reaffirm  that  critical  grade  controlling  duties  had  in  fact  been 
performed.  Another  way  to  deal  with  the  issue  would  be  to  tie 
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the  time-card  systea  into  the  personnel  systea,  and  provide  for 
tiae  keeping  on  a  task  basis.  If  tasks  did  not  Batch  the  grade 
level  of  the  eaployee,  a  flag  would  be  raised.  And,  of  course, 
follow  up  reviews  by  classifiers  could  be  conducted.  It  is 
obvious  that  this  is  a  very  critical  issue,  and  needs  to  be 
decided  early  on,  if  an  expert  systea  is  pursued. 

5.  Finally,  even  in  using  this  very  liaited  tool,  I  was 
encouraged  that  expert  systeas  technology  can  indeed  be  applied 
to  the  personnel  adainistrat ion  business.  It  is  as  if  aajor 
parts  of  our  business  were  Bade  for  this  technology.  Much  of 
what  we  do  is  rule  intensive,  and  so  are  expert  systeas.  Most  of 
what  we  deal  with  is  non-nuaeric  data,  and  expert  systeas  handle 
this  kind  of  inforaation  well.  Experts  in  our  business  do  exist, 
and  they  can  vocalize  their  expertise,  which  expert  system 
builders  need. 

The  deaonstration  aodule,  while  very  liaited  in  scope,  allowed 
for  a  trial  application  of  expert  systea  developaent.  It  can  be 
used  to  deaonstrate  the  technology,  and  has  provided  insight  to 
this  researcher  into  soae  of  the  techniques  used,  and  problems 
that  can  be  encountered. 

The  coaplete  set  of  aodels  are  contained  in  Appendix  B.  They  are 
printed  in  Expert  Ease  foraat.  Each  aodel  contains  the 
attributes  file  with  the  questions  that  will  be  asked  by  the 
systea,  the  exaaples  file  which  shows  the  exaaples  used  to  induce 
the  rules,  and  a  rule  file  which  is  the  decision  tree  used  by  the 
systea  to  provide  an  answer. 

It  is  not  possible  to  provide  the  floppy  disk  containing  the  demo 
aodule  with  any  except  the  original  copy  of  this  report.  A  copy 
of  Bxpert  Base  is  required  to  run  the  aodule.  Bxpert  Ease  is 
copyrighted  and  can  not  copied  and  further  distributed  (besides 
it’s  copy  protected,  too). 

An  exaaination  of  the  ability  of  this  demonstration  module  to 
accurately  classify  jobs  in  a  real-world  setting  was  undertaken. 
Results  of  this  test  are  suaaarized  in  Table  1.  The  method  used 
was  to  have  a  supervisor  use  the  systea  to  classify  an  actual  job 
over  which  the  supervisor  was  responsible  and  knowledgeable. 
These  tests  were  conducted  at  the  US  Aray  Yuma  Proving  Ground, 
AZ.  The  test  plan  was  initially  designed  to  compare 
classification  results  of  the  deaonstration  module  against  a 
huaan  classifier  in  terms  of  accuracy  and  time  expended  making 
the  classification  judgaent.  The  plan  was  modified  after  it  was 
discovered  that  aany  of  the  jobs  contained  mixed  duties,  i.e.  not 
purely  of  one  occupational  category. 

It  was  also  discovered  that  soae  of  the  questions  contained  in 
the  module  needed  to  be  adjusted  in  order  to  more  accurately 
apply  expert  judgaent,  and  remove  from  the  questioning  process 
any  opportunity  for  the  user  of  the  system  to  apply  judgment. 
For  exaaple,  the  aodule  that  classifies  crane  operator  jobs 
contains  a  question  about  precision  operation  of  a  crane.  The 
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question  is:  "Will  the  booa  crane  be  operated  in  any  of  the 
following  situations?  Load  and  unload  equipment  and  supplies  on 
board  ships.  Precision  set  objects  such  as  guns  into  mounts, 
sonar  equipment  on  to  ships,  or  wings  on  to  aircraft.  Dig  and 
move  earth  or  rock  close  to  buildings,  walls,  or  underground 
obstructions.  Destroy  brick  and  stone  structures  near  high- 
voltage  power  lines  or  other  structures."  The  supervisor  who 
used  this  module  answered  yes  to  this  question,  but  when  asked 
the  percent  of  time  spent  so  operating  the  crane  indicated  only 
5X.  The  end  result  was  a  Crane  Operator  W-09.  The  job  is 
classified  correctly  by  the  classifier  as  a  W-ll.  The  reason  for 
the  difference  lies  in  these  two  questions. 

When  the  supervisor  answered  the  question,  he  did  not  consider 
the  placing  of  gun  shelters  at  firing  positions  as  precision  use 
of  the  crane  as  defined  in  the  question.  Thus  a  large  part  of 
the  employee’s  work  time  was  not  included  in  the  following 
question  about  percentage.  The  classifier  who  had  classified  the 
job  had  made  the  judgment  that  placing  of  gun  positions  to  be 
precision  use  of  the  crane,  which  when  added  to  other  precision 
use  of  the  crane  resulted  in  at  least  25X  precision  operation. 
This  pointed  up  a  problem  with  the  module  which  did  not  solicit 
enough  information  about  crane  operation  to  make  a  judgment  about 
precision  use.  Therefore  more  development  of  the  module  would 
have  to  be  undertaken  to  rectify  this  problem.  A  similar  problem 
was  found  in  the  AC  mechanic  job.  In  this  case,  the  supervisor 
was  asked  to  make  a  judgment  about  mechanics  who  serviced 
equipment  that  had  a  direct  effect  on  tests.  The  module  needs  to 
be  modified  so  that  supervisors  respond  to  questions  non- 
judgmentall-y,  and  provide  for  the  judgment  to  be  made  by  the 
system. 

Table  1  shows  the  results  of  the  tests  conducted  at  Yuma  Proving 
Ground.  I  believe  the  tests  are  inconclusive  at  proving  or 
disproving  the  ability  of  the  demonstration  module  to  classify 
the  jobs  even  though  the  module  did  accurately  classify  6  of  8 
jobs . 


JOB 

COMPUTER 

CLASSIFIER 

Crane  Operator 

9 

11 

AC  Mechanic 

11 

10 

Motor  Vehicle  Operator 

7 

7 

Motor  Vehicle  Operator 

8 

8 

Mobile  Equip  Servicer 

5 

5 

Automotive  Worker 

8 

8 

Automotive  Mechanic 

10 

10 

Fork  Lift  Operator 

6 

6 

Table  1: 
Classifier. 


Test  Results  of  Demonstration  Module  vs.  Human 


The  value  of  these  tests  lie  in  the  kind  of  information  gathered 
from  the  supervisors  as  they  used  the  module,  rather  than  the 
accuracy  with  which  the  module  classified  the  jobs.  When  the 
system  provided  a  classification,  the  supervisors  felt  that  the 
system  did  not  ask  for  enough  information,  even  if  the 
classification  seemed  to  be  correct.  The  average  time  expended 
by  each  supervisor  on  each  job  was  about  3  minutes.  Much  of  this 
time  was  wasted  by  the  system  loading  files.  When  the  answer  was 
provided,  the  supervisor’s  reaction  was  invariably  "is  that  all?" 
This  reaction  is  attributed  to  the  past  experience  of  our 
supervisors  who  are  accustomed  to  being  asked  many  questions,  and 
showing  classifiers  what  the  job  is  all  about.  As  has  been  my 
experience,  classifiers,  for  good  reason,  gather  many  more  facts 
about  a  job  in  preparation  of  classifying  it  than  is  actually 
used  in  the  classification  process.  Many  times,  classifiers 
gather  these  non-essential  facts  in  order  to  give  supervisors  and 
employees  the  feeling  that  the  job  has  been  thoroughly  reviewed. 

It  may  be  necessary  to  build  an  expert  job  classification  system 
that  queries  the  supervisor  for  many,  many  facts,  even  though 
some  of  the  facts  are  non-critical  to  the  job  classification 
process.  This  may  be  particularly  true  in  the  transitional  phase 
over  several  years  until  our  supervisors  become  accustomed  to  the 
automated  approach  to  job  classification.  This  is  an  area  that 
requires  further  assessment  as  a  prototype  expert  system  is 
built . 
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There  are  nuaerous  areas  in  which  expert  systems  technologies 
could  be  put  to  productive  use  in  the  civilian  personnel 
administration  functions.  Practically  any  function  in  which 
personnel ists  apply  knowledge  to  the  solution  of  problems  is  a 
candidate  for  expert  systems.  As  discussed  in  the  previous 
chapter,  an  expert  system  could  be  developed  to  classify  jobs. 
Systems  could  also  be  built  to  make  qualifications 
determinations,  to  advise  on  health  benefit  plans  for  employees, 
to  advise  on  disciplinary  actions  (penalty  to  assess,  whether  a 
case  is  sufficient,  etc.),  to  make  determinations  for  non¬ 
government  training  approvals,  to  help  assess  negotiability 
issues,  to  make  pay  setting  determinations,  and  to  conduct  RIF’s. 
In  fact,  I  believe  there  are  few  areas  in  our  business  in  which 
expert  systems  could  not  be  put  to  use. 


Some  functions  performed  by  personnelists  could  not  be 
efficiently  handled  by  expert  systems.  These  are  invariably 
those  things  personnelists  do  that  are  intended  to  influence 
others  behavior.  For  example,  labor  negotiations  could  not  be 
handled  directly  by  an  expert  system.  Humans  are  definitely 
required  at  the  bargaining  table.  However,  expert  systems  could 
be  put  to  use  to  advise  the  human  negotiators  to  do  their  job 
better.  Employee  counseling  by  personnelists  in  which  behavior 
modification  is  a  goal  could  also  not  be  accomplished  by  a 
computer.  Those  activities  in  which  personal  interaction  is  a 
necessary  part  of  the  process  in  order  to  make  assessment  of  non¬ 
verbal  signals  such  as  body  language,  facial  expressions,  tone  of 
voice,  etc.  are  not  candidates  for  expert  systems. 

Those  functions  that  are  principally  application  of  rules, 
whether  the  rules  are  written  regulations,  or  general  principles 
learned  through  experience,  are  prime  candidates  for  this 
technology.  For  example,  the  determination  as  to  whether  or  not 
an  employee  has  retreat  rights  to  another  position  during  a  RIF 
is  based  on  the  application  of  several  rules.  One  must  determine 
if  the  position  in  question  is  one  from  or  through  which  the 
employee  was  promoted  or  which  is  a  position  that  is 
substantially  the  same  as  a  position  from  or  through  which  the 
employee  was  promoted.  These  determinations  are  sometimes  simple 
facts  (i.e.  the  employee  WAS  promoted  from  the  position),  and 
sometimes  judgments  (i.e.  substantially  the  same).  The  judgment 
whether  or  not  the  position  is  substantially  the  same  is  based  on 
an  expert  assessment  of  the  position  in  question.  The  same 
knowledges  used  by  the  expert  to  make  this  judgment  can  be 
captured  by  an  expert  system  which  could  make  the  same  judgment. 
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Not  only  could  this  judgment  be  made  by  the  expert  system,  if  the 
system  were  built  to  conduct  RIFs,  and  the  assessment  of  retreat 
rights  were  part  of  the  design,  retreat  rights  would  never  be 
simply  overlooked  by  the  expert  system  as  they  might  be  by  humans 
who  conducted  the  RIF. 

As  discussed  previously,  expert  systems  make  use  of  a  technique 
known  as  production  rules  that  take  the  form  of  IF  -  THEN.  In 
many  ways  these  rules  are  similar  to  the  rules  contained  in 
regulations  that  guide  the  operations  of  personnelists  working  in 
this  field.  Frequently,  regulatory  rules  require  interpretation 
by  personnelists  in  order  to  be  applied  correctly  to  a  given 
situation.  As  in  the  forgoing  example,  one  could  not  expect  any 
supervisor  or  employee  to  adequately  apply  the  "substantially  the 
same”  judgment  to  a  given  situation.  In  order  to  make  this 
determination,  a  personnelist  with  considerable  experience  must 
apply  his  or  her  expertise  to  the  problem,  and  make  the  call. 
What  the  personnelist  does  in  making  the  call  is  based  on 
gathering  additional  facts  (review  the  job  in  question)  and  apply 
rules  that  have  been  formed  through  experience.  These  rules  are 
the  knowledge  that  must  also  be  built  into  an  expert  system  in 
order  for  it  to  function  at  the  human  expert  level. 

Since  rule  based  techniques  are  used  to  implement  an  expert 
system,  then  an  approximation  of  the  number  of  rules  that  would 
be  required  will  help  to  estimate  other  things,  such  as  time  to 
develop,  size  of  the  system  hardware,  and  complexity  of  the 
software  tool  needed.  Therefore,  I  attempted  to  make  an 
estimation.  This  area  of  rules  estimation  is  not  well  understood 
even  by  the  experts.  It  seems  that  one  cannot,  in  advance, 
predict  with  any  degree  of  certainty  just  how  many  rules  will  be 
required  to  capture  the  expertise  in  any  given  knowledge  domain. 
The  field  is  so  new,  and  experienced  knowledge  engineers  so 
scarce,  little  has  been  done  to  develop  a  reliable  method  of 
estimating  rules. 

Review  of  existing  expert  systems,  and  the  number  of  rules 
involved  in  their  operation  does  not  shed  much  light  on  the 
problem.  Rl,  Digital  Equipment  Corporation’ s  computer  configurer 
has  varied  over  the  years  in  the  number  of  rules  it  has.  For 
example,  the  initial  system  had  777  rules,  and  recently  it  was 
reported  to  have  grown  to  3303.  Most  of  this  growth  was  due  to 
extending  its  capability  to  configure  additional  computers.  Over 
the  years,  it  has  had  ups  and  downs,  as  new  rules  are  created, 
and  others  dropped,  or  refined.  However,  it  is  not  possible  to 
evaluate  the  number  of  potential  rules  in  one  knowledge  domain 
based  on  any  other  knowledge  domain. 

If  one  were  to  count  only  the  rules  that  are  well  formed  in  the 
personnel  business,  then  it  would  be  a  fairly  simple  matter  to 
examine  the  regulation  involved  and  count  the  rules  it  contained. 
For  example,  the  rule  that  an  employee  must  have  a  college  degree 
in  library  science  as  a  prerequisite  to  qualifying  for  a 
librarian  job  is  relatively  well  formed.  However,  the  rules  that 
must  be  created  to  evaluate  even  general  experience  for  a 
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clerical  position  are  not  well  forned.  Staffing  specialists  are 
very  adept  at  reviewing  work  experiences  of  employees  and 
determining  whether  or  not  the  general  experience  requirements 
are  met,  and  they,  in  fact,  are  applying  rules  when  they  do  this. 
These  rules  of  thumb  based  on  years  of  experience,  are  the 
essence  of  the  knowledge  engineering  process  to  build  the  expert 
system.  Attempting  to  estimate  just  how  many  of  them  there  are 
for  any  specific  process  is  simply  not  a  very  precise  endeavor. 

My  methodology  for  making  an  estimate  was  based  on  a  rudimentary 
understanding  of  what  rules  look  like  in  expert  systems,  and  how 
they  are  used,  my  experience  in  the  knowledge  domain  of  personnel 
administration,  and  the  little  hands  on  development  of  the 
demonstration  module  which  classifies  eight  wage  grade  jobs.  I 
selected  five  white  collar  occupations,  examined  the  standards 
pertaining  to  them,  and  conducted  an  analysis  of  the  number  of 
apparent  rules  that  applied  to  making  decisions  about  various 
factors  contained  in  the  standards.  Searching  through  the 
standards  in  this  manner,  I  counted  the  number  of  what  appeared 
to  be  rules  that  would  have  to  be  written  into  an  expert  system 
in  order  to  make  decisions  about  the  various  elements  and  issues 
involved  in  solving  a  problem  of  deciding  the  grade  of  a  job. 
Figure  2  contains  the  estimated  number  of  rules  required  to 
classify  a  job  in  the  Budget  Analysis  Series,  broken  down  to 
rules  for  each  factor. 


Factor 

Rules 

Series  Determination 

25 

Knowledge  Required 

105 

Supervisory  Controls 

SO 

Guidelines 

50 

Complexity 

75 

Scope  and  Effect 

60 

Personal  Contacts 

15 

Purpose  of  Contacts 

20 

Physical  Demands 

1 

Work  Environment 

1 

Estimated  Total  432 

Figure  2:  Estimated  number  of  rules  for  classifying  Budget 

Analysis  Jobs. 


I  did  not  attempt  to  write  any  of  the  rules  that  appeared  to  be 
required  for  this  decision  process.  An  example  of  a  rule  under 
Personal  Contacts  might  look  like  this  (in  English):  "IF  there 
are  recurring  face-to-face  and/or  telephone  contacts  with  members 
of  Congress  and/or  top  Presidential  advisors  THEN  consider  a 
value  of  110  points  with  a  certainty  factor  of  .90".  Production 
rules  such  as  this  are  formed  in  English,  then  reduced  to  code  in 
the  language  being  used  to  build  the  system. 
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Figure  3 

gives  the  estimate  of  the 

number  of 

rules  that  might 

be 

required 

in 

the  five  occupational 

categories 

I  examined. 

The 

differences 

between  the  two  engineering 

occupations 

is 

attributable 

to  the  two  different  kinds  of  standards  involved. 

Occupation 

Rules 

Budget  Analyst 

432 

Mechanical  Engineer 

190 

Electrical /Electronic 

Engineer 

400 

Secretary 

367 

Supv  Grade  Eval  Guide 

(I  &  II) 

315 

Figure  3: 

Estimated  number  of  rules 

required 

to  classify  jobs 

in 

selected  occupational  categories. 


The  Mechanical  Engineer  standard  is  in  FES  format,  while  the 
Electrical  is  not.  Reflecting  on  this,  I  seemed  to  generate  a 
lot  more  rules  in  my  head  when  trying  to  make  distinctions  using 
the  narrative  format  standard.  It  is  also  interesting  to  note 
that  the  FBS  Budget  Analyst  jobs  resulted  in  a  number  of  rules 
similar  to  the  non-FBS  Electrical  Engineer.  I  can  ">nly  say  that 
the  job  of  trying  to  estimate  rules  is  extremely  hazardous  and 
speculate  that  it  might  take  about  400  rules  for  each  occupation 
to  be  classified. 

The'  estimates  are  for  job  classification  purposes  only.  My 
attempt  to  assess  the  number  of  rules  for  qualifications 
determinations  for  the  same  occupations  did  not  allow  me  to  use 
the  same  process  as  I  used  for  job  classification  purposes  since 
I  have  very  little  personal  experience  as  a  staffing  specialist. 
Using  somewhat  the  same  techniques,  however,  I  estimated 
approximately  200  rules  for  the  Budget  Analysis  Series.  This  did 
not  approximate  the  classification  rules  estimate,  so  since  I  did 
not  feel  I  had  the  requisite  background  in  staffing,  I  made  no 
further  attempts  to  get  a  handle  on  it  using  an  evaluation 
methodology.  It  might  be  safe  to  assume  that  the  two  are 
somewhat  equal  based  on  some  equality  in  the  amount  of  time  it 
takes  to  make  judgments  in  each  area  by  practitioners  of  the 
disciplines. 

This  process  of  rules  estimation  helped  me  to  form  the 
recommendation  in  this  report  that  the  next  step  in  examining  the 
application  of  expert  systems  to  the  personnel  business  involves 
the  development  of  a  prototype.  Accurate  assessment  of  the 
number  of  rules  required  for  any  specific  functional  area  can 
only  be  made  as  a  result  of  the  developmental  process. 

Expert  systems  can  definitely  be  used  in  CPO  functions.  How 
these  expert  systems  will  be  used  (i.e.,  by  whom)  is  explored  in 
the  next  chapter. 
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Most  of  the  expert  systems  that  were  developed  during  the 
gestation  period  of  this  technology  in  the  research  settings  of 
universities  were  designed  to  be  used  by  specialists  in  the 
knowledge  domain.  MYCIN  was  designed  to  be  used  by  doctors  who 
were  not  expert  at  diagnosing  infectious  diseases.  MYCIN* s  main 
usage  is  in  a  hospital  setting  in  which  the  patient  appears  to 
have  contracted  an  infectious  disease  while  in  the  hospital  for 
some  other  reason  (e.g.  surgery).  The  patient’s  doctor  can  use 
MYCIN  with  fragmentary  test  results  such  as  lack  of  cultures 
which  require  several  days  to  grow  in  order  to  make  positive 
diagnosis.  MYCIN  serves  as  an  expert  who  is  extremely  capable  of 
diagnosing  meningitis,  for  example,  without  the  full  complement 
to  cultures,  etc.  that  take  time  to  get.  The  terminology  used  by 
MYCIN  would  not  be  understandable  to  non-medical  personnel,  and 
could  not  be  used  by  them.  One  must  be  a  medical  doctor  to  use 
the  program.  PROSPECTOR  likewise  must  be  used  by  geologists 
since  the  terminology  used  would  not  be  understood  by  the  man  in 
the  street. 

The  design  .of  an  expert  system  for  a  knowledge  domain  in  the 
field  of  personnel  administration  must  first  be  assessed  in  terms 
of  who  would  use  it.  It  is  one  thing  to  build  a  system  to  be 
used  by  personnel  people  capable  of  making  some  judgments  and 
understanding  specialized  terminology,  and  it  is  another  to  build 
a  system  that  is  capable  of  advising  or  taking  action  based  on 
input  by  nearly  anyone.  For  example,  the  term  'planned 
management  action*  is  a  fairly  well  understood  phrase  by 
journeyman  level  personnelists .  The  phrase  is  much  less 
understood  by  supervisors  and  employees.  Therefore,  a  system 
that  has  been  designed  to  advise  personnelists  in  a  particular 
functional  area  could  refer  to  the  phrase  with  a  question  such  as 
"Is  this  the  result  of  a  planned  management  action?"  in  order  to 
gather  information  for  an  advisory  function  in  say  a  merit 
promotion  action.  However,  if  the  system  were  being  designed  to 
be  used  by  supervisors  or  employees  directly,  then  information 
would  have  to  be  gathered  in  another  manner  in  order  for  the 
system  to  make  the  initial  judgment  as  to  whether  or  not  a 
planned  management  action  had  been  taken  before  moving  on  to  the 
area  in  which  this  information  would  be  used. 

It  is  possible  to  design  an  expert  system  in  the  personnel 
business  to  serve  as  either  an  advisor  to  supervisors  and 
employees,  or  as  an  advisor  to  personnelists.  Obviously,  a 
system  designed  for  personnelists  use  would  be  much  smaller  due 
to  the  lack  of  need  to  have  it  make  interim  judgments  about 
common  terms  and  rules  well  understood  by  the  personnelist  users. 


There  are  soae  areas  in  which  such  systeas  would  be  the  natural 
choice.  For  example,  an  expert  system  designed  to  advise  a 
personnelist  on  negotiability  issues  to  be  used  at  the  bargaining 
table  could  be  built  without  ever  expecting  the  system  to  be  used 
by  employees  who  are  not  knowledgeable  practitioners  in  labor 
relations  matters. 

Other  areas  of  personnel  administration  do  not  offer  such  obvious 
choices.  Job  classification,  for  example,  is  -one  area  in  which 
the  system  could  serve  as  either  a  classier’s  expert  advisor  in 
grading  difficult  jobs,  or  it  could  serve  as  the  classifier 
proper,  and  grade  jobs  based  on  supervisor  or  employee  input. 
Since  supervisors  and  employees  are  knowledgeable  of  the  job,  its 
tasks  and  responsibilities,  it  appears  to  be  feasible  to  design 
an  expert  system  to  serve  as  the  classifier  (expert)  that  draws 
facts  and  information  about  a  job  from  a  supervisor,  and  makes 
the  classification  judgments  based  on  this  information.  It  is 
very  important,  and  a  considerable  challenge  to  use  carefully 
chosen  terminology  that  does  not  demand  of  the  user  a  knowledge 
of  the  classification  process,  nor  the  ability  to  understand 
specialized  terminology,  nor  the  requirement  to  make  judgments 
about  meaning  of  terms.  I  attempted  to  do  this  with  varying 
degrees  of  success  with  the  demonstration  module.  For  example, 
rather  than  asking  whether  the  crane  operator  was  required  to 
make  precision  placement  of  loads  with  a  crane,  which  would 
require  the  user  to  make  a  judgment  about  what  is,  or  is  not 
’precision’,  I  attempted  to  ask  questions  about  specific 
operations  which  were  known  by  the  module  to  be  precision  tasks, 
without  requiring  the. user  to  make  a  judgment.  The  design  of 
these  questions,  ridding  them  of  any  requirement  for  the  user  to 
make  judgments  about  the  meaning  of  terms,  is  probably  the 
biggest  challenge  to  developing  an  expert  system  to  be  used 
directly  by  supervisors  and  employees. 

While  this  challenge  is  great,  I  do  not  believe  it  to  be 
insurmountable.  Nearly  any  specialized  term  used  in  personnel 
jargon  can  be  "de-personnelized"  with  other  terms,  examples  or 
explanations . 

I  believe  a  goal  for  system  development  needs  to  be  set  to  design 
a  system  for  direct  supervisor  use.  This  belief  is  based  on 
practical  considerations.  While  there  are  some  areas  in  which 
expert  systems  could  be  put  to  use  in  a  personnel  office  that 
would  serve  as  experts  to  knowledge  workers  (personnelists) ,  the 
biggest  payoff  of  an  expert  system  would  allow  for  its  use 
directly  by  supervisors  and  employees.  In  this  manner,  actual 
savings  could  be  realized  by  replacing  personnelists  with  the 
expert  system. 

There  are  other  issues  that  relate  to  the  use  of  expert  systems 
in  this  manner  that  are  perhaps  more  difficult  to  accept  from  a 
policy  viewpoint  than  from  a  technical  consideration.  These 
involve  the  manner  in  which  personnel  administration  and 
management  operate  within  the  agency.  While  it  is  a  stated 
policy  that  the  supervisor  IS  the  personnel  manager  in  Army,  much 
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of  the  tiae  expended  in  a  personnel  office  by  personnelists  is 
ained  at  not  only  advising  the  supervisor  of  things  that  should 
or  should  not  be  done,  but  also  of  forcing  or  preventing  things. 
The  police  roll  of  personnelists  is  steeped  in  tradition,  and 
reinforced  through  various  influences  such  as  training  courses, 
inspections  and  surveys,  and  reporting  procedures.  Some  of  our 
functions  are  sore  heavily  engaged  in  police  work  than  others. 
For  exaaple,  Position  Managenent  and  Classification  is  at  one  end 
of  the  scale,  while  Training  and  Development  is  probably  at  the 
other.  With  an  expert  system  that  is  directly  operated  by 
supervisors  and  employees,  the  opportunity  for  supervisors  to 
circumvent  the  police  roll  personnelists  play  is  increased. 

I  attempted  to  get  this  issue  addressed  by  some  of  our  most 
highly  placed  personnelists  in  Army  through  a  questionnaire  that 
was  handed  out  at  the  Professional  Development  Seminar  on  2  May 
1985,  in  the  Pentagon.  Approximately  50  key  professionals 
attended  the  seminar  and  were  asked  to  respond  to  a  questionnaire 
which  had  the  issue  of  direct  supervisory  use  of  expert  systems 
identified  (the  questionnaire  is  at  Appendix  I).  Of  those 
attending  only  three  responded  to  this  questionnaire.  During  the 
seminar,  three  groups  were  formed  to  discuss  the  concepts  and 
issues  of  experts  systems,  and  a  report  of  their  deliberations 
which  covered  most  of  the  issues  identified  in  the  questionnaire 
was  provided  to  DA.  Those  reports  are  enclosed  at  appendix  I.  I 
also  handed  out  the  questionnaire  after  giving  a  briefing  on 
expert  systems  to  attendees  of  the  Western  Area  CPO  conference 
held  on  13  May  1985  in  the  San  Francisco  Field  Office.  Of 
approximately  20  attendees,  only  three  responded.  These 
responses  are  summarized  at  Appendix  I. 

I  have  interpreted  the  responses,  and  non-responses,  from  these 
attempts  to  get  the  issues  addressed  as  a  "wait  and  see" 
attitude.  Other  interpretations  could  be  made  such  as 
"indifference",  or  "don’t  understand",  but  I  believe  the  correct 
one  is  generally  cautious  observation  from  the  sidelines.  This 
is  not  unusual.  It  has  been  observed  by  others  that  most  new 
ideas  are  not  immediately  embraced,  nor  totally  rejected  upon 
first  introduction. 

In  any  event,  the  issue  of  who  the  intended  user  of  an  expert 
system  will  be  needs  to  be  decided  early  on.  One  cannot  begin 
development  until  one  knows  who  will  use  the  system.  The  user 
interface  is  a  key  ingredient  in  any  expert  system,  and  is 
developed  along  with  the  the  knowledge  base  and  inference  engine. 

Recommend  the  targeted  user  of  the  expert  system  prototype  to  be 
proposed  in  a  later  chapter  be  supervisors  and  employees.  This 
will  give  the  greatest  potential  for  savings. 
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One  of  the  aajor  tasks  undertaken  in  this  project  was  an 
assessment  of  whether  expert  systems  could  be  used  in  civilian 
personnel  adainistration  functions,  and  whether  they  should  be. 
Five  aajor  hypotheses  were  formed  as  a  result  of  ay  assessment, 
and  will  be  presented  in  this  chapter  of  the  report. 

Hypothesis  1.  Expert  systems  technology  can  be  applied  to 
civilian  personnel  adainistration  functions.  My  conclusion  on 
this  hypothesis  is  affirmative,  and  is  based  on  the  following 
findings . 

First  of  all,  auch  of  the  civilian  personnel  adainistration 
business  is  a  knowledge  intensive  activity.  That  is  to  say,  the 
majority  of  things  that  are  done  by  personnelists  is  based  on  a 
highly  specialized  area  of  knowledge.  Classifiers  deal  daily 
with  knowledge  about  jobs,  e.g.  what  are  typical  and  non-typical 
tasks  found  in  various  jobs.  They  apply  knowledges  about  how 
jobs  are  classified,  what  constitutes  a  aajor  duty,  how 
organizations  are  efficiently  structured  and  so  forth.  Staffing 
specialists  deal  with  knowledges  about  aaount  •  and  kinds  of 
experience  and  training  that  qualify  people  for  jobs,  what  the 
potential  for  filling  various  jobs  is,  how  to  set  pay  and  so 
forth.  Most  of  our  daily  work  is  knowledge  intensive  as  opposed 
to  skills  intens-ive  such  as  operating  a  piece  of  equipment. 

Expert  systeas  technology  directly  applies  to,  and  works 
extremely  well  with  knowledge  intensive  doaains.  Expert  systems 
are  frequently  called  knowledge  based  systems.  Thus  it  can  be 
expected  that  our  knowledge  intensive  activities  are  ideal 
candidates  for  expert  systeas  applications. 

Second,  personnel  adainistration  is  a  rules  oriented  function. 
Our  knowledge  is  based  heavily  on  applying  rules  to  specific 
problem  situations.  Many  of  our  rules  are  highly  structured  and 
in  fact  written  in  the  fora  of  regulations,  manuals,  policies  and 
the  like.  This  is  not  to  say  that  ability  to  interpret  rules, 
and  convince  others  of  their  acceptance  is  not  important. 
Interpretation  is  another  fora  of  knowledge  application,  while 
the  ability  to  convince  others  of  their  acceptance  is  a  skill. 
However,  the  accurate  application  of  rules  to  problem  situations 
is  a  very  aajor  part  of  any  personnel ist ’ s  daily  activities. 

Expert  systems  are  very  well  suited  for  knowledge  domains  in 
which  rules  are  well  formed  and  play  an  important  part  in  the 
function.  Many  expert  systems  are  dependent  on  rules  for  their 
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proper  functioning.  Using  IF  -  THEN  production  rule  constructs, 
expert  systems  technology  fits  well  with  our  heavily  rules 
oriented  activities. 

I  have  therefore  concluded  that  much  of  our  personnel  business 
seems  highly  compatible  with  this  technology,  and  that  expert 
systems  could  accomplish  many  tasks  performed  by  knowledge 
workers  in  this  domain. 

Hypothesis  2.  If  expert  systems  technology  is  applied  to  our 
knowledge  domain  where  applicable,  then  we  can  save  money  for  the 
American  taxpayer. 

It  is  possible  to  develop  an  expert  system  that  can  be  used  by 
persons  without  specialized  knowledges  of  the  domain. 
Financial  institutions  are  developing  systems  to  be  used  by  non¬ 
specialist  personnel  to  make  analysis  for  granting  loans, 
marketing  departments  are  developing  systems  to  advise  and  train 
salespersons  in  new  products  and  configuration  technicalities, 
and  other  similar  developments  are  under  way.  An  expert  system 
could  be  developed  to  be  used  directly  by  supervisors  to  provide 
advice  and  take  actions  that  personnel  people  now  do.  In  this 
manner,  the  number  of  specialized  knowledge  workers  could  be 
reduced  since  the  expert  system  is  performing  many  of  the  their 
tasks.  A  gross  approximation  of  possible  savings  appears  in 
another  chapter  of  this  report. 

I  have  concluded  that  cost  savings  could  result  with  the 
application  of  expert  systems  by  reducing  the  number  of  knowledge 
workers  in  the  civilian  personnel  functions. 

Hypothesis  3.  If  an  expert  system  is  put  into  place, 
responsibility  and  accountability  for  personnel  management  can  be 
fixed  directly  with  whom  current  Army  policy  states  it  belongs  - 
the  supervisor. 

With  an  expert  system  that  is  being  directly  operated  by 
supervisors  to  accomplish  personnel  actions,  the  supervisory 
input  to  the  system  is  the  immediate  and  responsible  cause  of  the 
system’s  resultant  activities.  Given  the  assumption  that  the 
system  is  performing  as  a  human  expert  would  perform  with  the 
same  facts,  any  inappropriate,  irregular  or  illegal  results 
produced  by  the  system  would  be  attributable  to  false  facts. 
Since  critical  facts  are  provided  by  the  supervisor  user,  the 
supervisor  assumes  the  full  responsibility  and  accountability  for 
the  outcome. 

My  conclusion  is  that  an  expert  system  insures  accountability  and 
responsibility  in  the  knowledge  domain  in  which  it  is 
functioning. 

Hypothesis  4.  Improved  consistency  of  personnel  administration 
on  an  Army-wide  basis  will  result  from  the  application  of  expert 
systems  technology. 


30 


This  is  fairly  obvious.  After  a  system  is  developed,  certified 
expert,  and  placed  in  operation  as  a  standardized  system, 
identical  results  would  be  obtained  given  identical  facts  in  all 
locations  throughout  Army.  Although  we  do  not  like  to  admit  it, 
human  specialists  do  not  always  provide  the  same  degree  of 
consistency.  One  could  argue  that  consistency  is  not  supreme, 
and  even  argue  that  what  might  be  lost  with  machine  performance 
(personal  contact  and  human  involvement)  is  not  preferable  to 
some  inconsistency.  I  will  not  attempt  to  debate  that  issue  in 
this  report. 

My  conclusion  is  that  consistency  of  results  is  maximized  with 
the  use  of  expert  systems  performing  functions  now  performed  by 
personnelists . 

Hypothesis  5.  Timeliness  of  service  provided  in  personnel 
administration  matters  will  be  orders  of  magnitude  better  than 
present  performance. 

First,  computers  are  simply  faster  than  humans  at  some  tasks, 
particularly  difficult  ones.  In  some  cases,  humans  do  outperform 
machines,  particularly  in  areas  of  cognition.  There  will  be  some 
tasks  in  which  the  expert  system  will  not  be  a  match  for  a  human 
expert.  Most  actions  that  are  taken  by  personnelists  throughout 
Army  are  not  being  taken  by  experts,  but  rather  specialists.  If 
all  of  our  specialists  were  experts,  perhaps  humans  would 
outperform  machines.  On  the  whole,  however,  computers  will  be 
able  to  accomplish  tasks  faster  than  is  currently  possible  with 
our  personnel  specialists. 

Secondly,  and  probably  most  importantly,  there  is  no  wait  time 
with  an  expert  system.  Much  of  the  total  processing  time  of  any 
action  under  our  current  system  is  actually  wait  time  in  which  no 
one  is  doing  anything  with  it.  It  is  laying  in  an  in-box.  Not 
because  there  is  a  human  sitting  there  doing  nothing  while  the 
action  sits,  but  because  the  human  is  doing  something  else.  With 
an  expert  system,  wait  times  are  measured  in  micro-seconds,  not 
days.  With  the  system  available  at  the  work  site  of  the 
supervisor,  action  would  be  taken  immediately  by  the  computer  to 
respond  to  the  supervisor  and  begin  the  processes  of  solving  the 
problem  or  taking  the  action.  State-of-the-art  multiuser 
computer  systems  share  CPU  time  so  efficiently  that  the  user  is 
unaware  that  the  system  is  responding  to  more  than  one  user. 
Humans  simply  can't  do  this. 

My  conclusion  is  timeliness  would  definitely  be  greatly  improved 
in  any  area  in  which  expert  systems  are  appropriately  applied. 

These  conclusions  not  only  indicate  that  expert  systems  can  be 
applied  in  our  business,  but  that  perhaps  they  should.  At  least 
there  is  enough  positive  evidence  that  they  should,  that  I 
believe  the  next  step  needs  to  be  taken.  Some  of  the  issues 
cannot  be  realistically  assessed  without  a  system  in  operation. 
For  example,  it  is  prue  conjecture  whether  or  not  supervisors 
will  try  to  cheat  the  system,  given  the  chance.  The  next  step  is 
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the  prototype  development  stage.  The  scope  of  the  system  needs 
to  be  defined,  a  prototype  developed,  refined,  tested  and  placed 
into  operational  test  mode  in  parallel  with  the  current  way  of 
doing  business  and  assessed.  This  will  be  discussed  in  the  next 
chapter . 
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Expert  systems  technology  can  definitely  be  employed  in  Army’s 
civilian  personnel  administration  functions.  It  is  therefore 
recommended  that  the  next  stage  of  development  be  undertaken. 
During  this  stage,  a  prototype  system  will  be  developed.  This 
section  will  delineate  what  the  expert  system  would  do,  and  not 
do  as  a  result  of  this  developmental  stage.  It  is  an  outline 
that  could  serve  as  a  guide  for  the  project  development  team.  It 
must  be  recognized  that  a  project  office  would  refine  the  scope, 
functions  and  details  of  the  expert  system  as  it  is  developed, 
thus  the  following  outline  would  expectedly  be  deviated  from  as 
the  system  is  developed,  and  more  experience  is  gained  with  what 
is  and  is  not  possible. 

It  is  proposed  that  the  demonstration  project  be  limited  to  the 
PM&C  function  of  personnel.  As  discussed  elsewhere  in  this 
report,  the  number  of  rules  associated  with  the  entire  personnel 
administration  function  is  astronomical.  Trying  to  develop  an 
expert  system  from  scratch  that  would  encompass  all  functional 
areas  of  personnel,  at  one  time,  is  simply  too  large  of  an 
undertaking  to  consider.  It  would  literally  take  years  to 
develop,  and  with  regulations  constantly  changing,  it  may  never 
be  completed,  and  placing  it  into  operation  for  test  purposes 
could  not  be  accomplished.  I  would  therefore  recommend  a  modular 
approach  to  development,  utilizing  the  PM&C  function  as  the  area 
to  test  first.  When  this  results  in  a  successful  expert  system, 
then  other  functions  could  be  added  incrementally.  In  this 
manner,  the  entire  array  of  civilian  personnel  administration 
functions  could  eventually  be  integrated  into  one  cohesive 
system. 

This  recommendation  did  not  come  easily.  During  my  research,  I 
continued  to  hold  on  to  the  idea  to,  at  a  minimum,  bring  the 
three  major  functions  of  classification,  staffing  and  performance 
management  under  a  test  program,  and  limit  the  size  of  the 
organization  being  serviced  so  that  it  would  be  small  enough  for 
the  system  to  handle.  The  reason  for  trying  to  maintain  this 
concept  is  due  to  my  belief  that  these  three  functions  are 
inextricably  intertwined,  and  should  be  considered  as  a  whole  by 
supervisors  and  employees  as  well  as  personnelists .  I  believe  we 
have  an  artificially  segmented  system  when  we  cause  supervisors 
to  visit  one  set  of  specialists  to  get  a  job  established 
(classified),  another  to  define  the  criteria  by  which  candidates 
will  be  evaluated  for  potential  placement  in  the  job,  and  yet 
another  set  of  specialists  to  identify  the  performance  criteria 
by  which  the  employee  will  be  measured.  In  each  functional 
specialty,  many  of  the  same  factors  are  discussed,  examined, 


defined  and  described,  with  a  slightly  different  angle  being 
explored  for  the  factor.  It  seems  efficient  to  me  that  the 
discussing,  examining,  defining  and  describing  of  these  factors 
should  be  done  all  at  the  same  time  with  the  slight  change  in 
angle  being  satisfied  for  each  specialty. 

I  feel  strongly  enough  about  this  opinion  that  I  held  on  to  the 
integrated  concept  for  as  long  as  I  could.  However,  I  eventually 
had  to  drop  the  idea  in  favor  of  actually  developing  a 
prototype  system.  I  made  the  assumption  that  any  expert  system 
would  have  to  closely  mimic  our  current  system  in  practice.  The 
sheer  size  and  complexity  of  each  functional  specialty  varies. 
For  example,  the  staffing  function  has  literally  scores  of 
regulatory  and  policy  issuances  that  come  to  bear,  some  on  nearly 
every  action,  and  some  very  rarely.  All  of  the  rare  rules 
applications  would  have  to  be  built  into  a  system  in  order  for  it 
to  replace  the  knowledge  workers  involved  in  staffing  functions 
and  mimic  our  current  system.  While  this  is  definitely  possible, 
the  time  it  would  take  to  develop  an  expert  system  that  covered 
the  entire  staffing  function  AND  the  classification  function 
would  more  than  double  the  time  it  would  take  to  produce  a  system 
covering  only  the  PM&C  function.  The  performance  management 
function,  with  little  real  expertise  currently  in  existence  (in 
comparison  with  Staffing  and  Classification)  would  probably  be 
the  easiest  and  quickest  expert  system  to  develop. 

I  have  also  assumed  that  Army  would  expect  a  system  to  be  placed 
into  operation  for  test  purposes  within  about  three  years  of  its 
approved  go  ahead.  To  spend  any  more  time  in  the  development 
phase  of  a  project  without  bringing  it  to  beta  testing  and  then 
on  line  use  would  be  unreasonable. 

Some  of  the  things  I  considered  which  caused  me  to  recommend  PM&C 
rather  than  R&P  or  some  other  function  are  as  follows. 

Position  classification  expertise  is  a  relatively  logical  and 
structured  knowledge  domain.  I  believe  rules  could  be  generated 
with  expert  system  development  tools  that  would  result  in 
accurate  classifications,  and  would  be  acceptable  to  the  experts 
knowledgeable  in  this  domain.  The  user  interface,  i.e.  the  part 
of  the  program  that  queries  supervisors  and  employees  for  input, 
appears  to  be  a  problem  that  can  be  solved,  and  would  not  involve 
an  inordinately  different  problem  from  the  knowledge  engineering 
process  itself.  What  this  means  is,  as  the  knowledge  engineering 
is  accomplished  to  "teach"  the  system  its  expertise,  the  user 
interface  closely  parallels  and  resembles  the  rules  that  would  be 
written.  By  comparison,  this  is  not  necessarily  as  close  a 
parallel  in  such  things  as  candidate  evaluation,  therefore  a 
separate,  but  related  programming  effort  would  be  required  to 
generate  a  user  interface  for  such  functions. 

Another  consideration  is  the  resultant  system  in  an  operational 
mode.  While  it  is  most  desirable  to  have  the  system  available  on 
site  at  every  work  place,  this  simply  may  not  be  possible  for  the 
test  phase  of  the  prototype.  Very  few  fully  distributed  ADP 
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networks  exist  in  the  Amy  today,  in  fact  none  that  I  could  find. 
Although  Most  installations  have  considerable  automation  efforts 
under  way,  I  would  not  expect  full  networks  available  to  all 
supervisors  at  many  activities  by  the  time  this  system  is  ready 
for  operational  purposes.  Given  these  circumstances,  the  PM&C 
module  is  more  amenable,  than  other  personnel  functions,  to  being 
made  available  at  a  few  sites  at  an  installation,  or  only  in  the 
personnel  office.  Supervisors  could  then  visit  the  location  of 
the  system  and  obtain  PM&C  services.  This  should  not  cause  a 
traffic  jam  since  the  frequency  with  which  supervisors  initiate 
new  PM&C  actions  that  require  full  classification  services  is  not 
that  great  (at  least  in  comparison  with  the  R&P  services). 

Given  the  above  assumptions  and  considerations,  I  recommend  the 
prototype  system  be  developed  for  the  Position  Management  and 
Classification  function  in  personnel. 

The  development  of  the  PM&C  expert  system  should  insure  that  it 
accomplishes  all  of  the  tasks  now  performed  by  classifiers  and 
others  assigned  to  a  typical  civilian  personnel  office  PM&C 
function.  While  it  may  seem  superfluous  to  specify  these  tasks 
for  the  benefit  of  personnel ists ,  I  am  including  a  summary  of 
these  tasks  for  those  who  might  read  this  report  that  are  not 
personnel ists ,  and  to  provide  the  outline  for  the  development  of 
the  system. 

The  test  system  would  be  equipped  to  classify  all  positions  in 
the  organizations  serviced.  The  method  used  would  be  direct 
supervisory  input  to  the  system,  with  the  system  prompting  for 
the  required  information  to  make  an  accurate  classification,  of 
any  job  in  the  organizations.  The  system  must  be  "smart"  enough 
to  recognize  when  it  is  dealing  with  a  job  for  which  it  cannot 
provide  a  classification,  and  report  same  to  the  user,  as  well  as 
notify  people  in  the  personnel  office.  Thus,  for  example,  if  the 
system  is  designed  to  service  a  Comptroller,  a  Procurement 
Office,  and  a  Personnel  Office,  and  a  serviced  supervisor 
developed  a  need  for  an  engineering  technician  that  was  not 
projected  by  the  system  developers  as  a  probable  occupation  in 
any  of  these  functions,  the  system  would  recognize  that  it  could 
not  classify  such  a  job,  and  make  appropriate  notifications. 
This  job  would  then  have  to  be  "person  classified"  outside  the 
scope  of  the  system. 

Mixed  jobs  must  be  acceptable  to  the  system,  as  long  as  the  tasks 
relate  to  jobs  for  which  the  system  has  been  developed.  The 
system  should  be  equipped  to  make  FLSA  determinations  on  all  jobs 
it  can  classify.  It  should  also  assign  competitive  level  codes, 
and  make  functional  classifications  if  engineering  positions  are 
included.  It  would  also  have  to  produce  correct  supervisory  and 
non-supervisory  indicator  codes. 

The  system'  would  prepare  a  job  description  based  on  the 
interactive  process  with  the  supervisor.  The  job  sheet  would  be 
printed,  and  an  evaluation  rationale  also  produced.  The 
evaluation  rationale  would  essentially  be  a  listing  of  the  rule 
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firings  that  were  used  internally  to  make  the  classification 
judgment.  It  would  also  be  a  record  of  supervisory  responses  to 
questions  posed  by  the  system,  thus  providing  an  audit  trail  of 
the  supervisory  input.  This  may  help  to  keep  supervisors  honest. 

The  system  would  also  perform  maintenance  reviews.  This 
function  would  be  performed  by  the  system  prompting  for 
information  from  both  the  supervisor,  and  the  employee  on  the 
job.  Inconsistencies  of  information  provided  by  each  source 
would  require  that  they  be  resolved  before  the  system  would 
verify  accuracy  of  the  job.  For  example,  if  the  supervisor 
specifies  that  a  particular  duty  is  still  required  in  a  job,  and 
the  employee  replies  that  he/she  has  not  performed  the  duty  in 
the  last  year,  this  inconsistency  would  have  to  rectified,  and 
the  system  would  require  its  resolution. 

Hazard  pay  and  environmental  differential  determinations  would  be 
made  by  the  system.  This  would  be  an  on-line  service  in  which 
supervisors  or  employees  who  feel  work  being  performed  warrants 
HDP/EDP,  could  query  the  system  for  advice  and  determinations  as 
to  whether  the  premium  pay  is  warranted.  The  system  should  be 
capable  of  making  a  determination,  issuing  a  certification  of 
work  judged  to  meet  criteria  for  payment,  and  notify  all 
appropriate  offices  and  persons. 

Position  management  advice  and  recommendations  should  also  be  a 
service  provided  by  this  system.  This  will  provide  the 
opportunity  to  tie  some  functions  into  the  system  that  are  not 
traditionally  the  sole  responsibility  of  the  personnel  office  (at 
least  in  most  personnel  offices).  For  example,  functional 
alignment  among  various  offices,  work  measurement,  costs 
considerations  and  the  like. 

The  system  must  have  in  its  knowledge  .base  approved  functional 
assignments  of  the  organizations  at  the  activity  in  which  it  is 
installed  (including  those  not  serviced  by  the  system).  This 
will  allow  for  the  system  to  check  for  inconsistencies  in 
functional  responsibilities  among  offices,  and  allow  for  advice 
to  be  given  in  cases  of  overlapping  functions. 

The  system  must  be  able  to  offer  advice  (and  perhaps  provide  some 
control)  on:  layering  of  supervision;  dilution  of  job  controlling 
duties  among  several  positions;  supervisory  ratios;  assistants 
and  deputies;  high  grade  considerations;  average  grade  trends, 
and  other  such  position  management  concerns. 

In  the  case  of  upgrades  on  jobs,  the  system  must  make  accurate 
judgments  on  job  reconstitution,  and  allow  coordination  with 
staffing  specialists  in  the  final  determination  of  non¬ 
competitive  promotions. 

As  mentioned  above,  the  actual  development  process  of  the  expert 
system  will  delineate  capabilities  that  must  be  built  into  it.  I 
strongly  urge  that  a  development  office  keep  in  mind  that  the 
system  must  completely  perform  all  tasks  that  would  be  assigned 


to  a  classifier.  Therefore,  design  limitations  of  the  system 
must  be  the  variety  of  occupations  it  is  equipped  to  handle 
initially,  rather  than  the  full  breadth  of  tasks  that  are 
performed  by  classifiers.  If  this  approach  is  taken,  the  only 
performance  limitation  of  the  system  would  be  the  number  of  jobs 
it  could  classify,  which  could  be  added  to  as  the  system  grows. 
This  would  insure  a  best  case  test  of  the  technology,  and  prove 
its  worth  as  a  people  replacement. 

In  order  to  keep  the  development  time  of  the  prototype  within 
reasonable  bounds,  I  recommend  the  system  be  initially  designed 
to  provide  PM&C  services  to  one  or  two  organisations  within  an 
installation  with  from  20  to  50  occupational  categories 
represented.  Perhaps  a  Comptroller,  or  a  Facility  Engineer,  or  a 
Procurement  Office  would  be  appropriate.  The  development  group 
will  need  to  determine  not  only  what  occupational  categories  are 
currently  present,  but  also  occupations  that  may  be  required,-  and 
in  the  case  of  mixed  jobs,  those  functions  performed  which  fall 
into  other  occupational  categories  but  are  not  classification 
controlling,  and  are  therefore  not  clearly  evident  from  a  review 
of  the  existing  occupations. 

After  the  prototype  is  developed,  it  will  be  tested  within  this 
organization,  so  the  organization  will  need  to  be  willing  to 
participate  in  the  test.  Before  the  organization  is  selected, 
its  supervisors  and  employees  will  need  to  be  thoroughly  briefed 
on  the  purpose  and  intent  of  this  development  project.  There 
should  be  no  attempt  to  "fix"  the  outcome  of  the  test  by 
selecting  oply  an  organization  that  wholeheartedly  agrees  with 
the  concept  of  expert  system  servicing,  and  by  the  same  token 
an  organizations  should  not  be  selected  that  is  openly  negative 
about  the  possibilities. 

The  number  of  employees  in  the  organization  is  of  little 
importance.  There  should  be  several  supervisors,  however,  which 
would  indicate  an  organization  of  at  least  50  people. 

I  believe  the  development  of  the  prototype  should  be  conducted  at 
the  installation  in  which  it  will  be  tested.  This  will  give  the 
development  team  the  opportunity  to  interact  freely  and  easily 
with  the  personnel  office,  the  target  organization,  and  other 
support  functions.  The  team  must  be  careful  not  to  allow  local 
unique  environmental  factors  to  influence  the  development 
process.  For  example,  personalities,  local  political 
relationships,  purely  local  policies  and  the  like  should  not  be 
of  any  influence  in  the  system’s  design. 


How  could  Army  go  about  building  the  prototype  expert  system? 
There  are  several  approaches  that  could  be  taken,  but  I  have 
boiled  them  down  to  two  basic  alternatives.  Both  of  these 
alternatives  will  be  described,  and  a  recommendation  will  be  made 
for  the  one  I  feel  has  the  most  merit. 

Alternative  #1. 

This  alternative  would  have  the  expert  system  built  by  a 
contractor  under  supervision  from  a  project  office  staffed  with 
personnelists.  The  government  would  develop  a  scope  of  work  for 
the  contract,  use  competitive  bid  procedures,  let  the  contract, 
administer  the  contract  as  the  system  is  built  by  the  company, 
and  place  the  system  into  operation  at  a  demonstration  project 
site.  The  project  office  would  assess  the  capability  of  the 
system  during  the  operational  phase,  either  make  adjustments  to 
the  system  directly,  or  continue  to  work  with  the  contractor  to 
have  him  fine  tune  the  system. 

There  are  companies  who  specialize  in  building  expert  systems  for 
customers.  I  have  contacted  several  of  them,  and  feel  that  an 
expert  system  could  be  built  using  this  approach.  There  are  pros 
and  cons  to  utilizing  this  strategy  which  will  be  discussed 
below. 

A  strawman  scope  of  work  statement  was  prepared  for  use  in 
soliciting  bids  from  contractors  to  develop  an  expert  system  if 
this  approach  were  selected.  This  statement  is  at  Appendix  K. 
The  statement  is  intentionally  non  specific  in  regard  to  the 
exact  processes  that  are  used  in  personnel  administration.  I 
first  began  to  spell  out  in  detail  the  processes  the  system  must 
mimic.  For  example,  to  spell  out  specifically  how  the  system 
must  classify  a  job,  and  which  jobs  it  must  classify.  However, 
this  activity  gets  into  extreme  minutia,  that  could  make  a 
statement  of  work  read  like  the  entire  FPM,  AR’s,  etc.  It  also 
gets  into  judgmental  areas  for  which  true  experts  need  to  be 
consulted  in  order  to  specify  the  parameters  for  expert  system 
operation.  It  also  depends  on  such  things  as  which  activity  will 
be  serviced  by  the  system  which  has  not  been  decided.  Therefore, 
the  statement  of  work  is  designed  to  specify  outcomes  of  what  the 
system  must  provide.  Essentially,  such  a  statement  could  have 
read  "Provide  an  expert  system  that  exactly  mimics  current 
functions  performed  by  a  journeyman  position  classification 
specialist".  However  it  was  felt  that  a  scope  of  work  would  have 
to  be  a  little  more  specific  than  that  in  order  for  a  contractor 
to  have  some  idea  of  what  it  meant.  The  scope  of  work  first 


needs  to  be  agreed  upon  that  it  encompasses  the  functions  to  be 
included,  and  then  a  consultation  is  required  with  procurement 
personnel  to  insure  it  meets  the  standard  of  adequacy  necessary 
to  solicit  bids. 

Estimation  of  time  frames  and  resources  needed  for  the 
demonstration  project  to  be  brought  on  line  in  an  operational 
mode  using  this  alternative  for  development  has  been  a  very 
difficult  area  to  assess.  My  estimates  herein  are  based  on 
interviews  with  knowledge  engineers  and  others  where  expert 
systems  have  been  developed,  and  certain  assumptions  about  the 
demonstration  project  installation  such  as  the  number  and  variety 
of  occupations  involved,  the  amount  of  distributed  ADP  hardware 
(usable  computer  terminals)  already  in  place,  etc.  The  estimate 
is  also  based  in  part  on  the  contractor’s  first  phase  report  on 
the  expert  system  being  developed  for  military  personnel 
functions  under  contract  DAAK1 1-84-C-0083  (US  Army  Ballistic 
Research  Laboratory) .  Assuming  the  scope  of  the  system  is  as 
described  above,  a  very  rough  estimate  of  $1.5M  to  $2.0M  has  been 
made  for  the  contract  to  develop  the  prototype  system. 
Development  time  would  be  in  the  neighborhood  of  two  to  three 
years . 

A  project  office,  directly  under  DA  oversight  will  be  required. 
The  project  office  must  have  a  relatively  direct  link  to  the 
Director  of  Army  Civilian  Personnel.  This  direct  link  is 
essential  due  to  the  importance  of  this  demonstration  project  to 
the  Army  as  a  whole,  and  the  visibility  it  will  receive  from  OPM, 
0MB,  and  perhaps  even  the  Congress.  Due  to  the  recommended 
installations  (White  Sands  Missile  Range,  Sacramento  Army  Depot, 
or  North  Pacific  Division,  COE),  all  of  which  are  in  the  Western 
US,  it  is  recommended  the  project  office  report  to  the  Field 
Office  Representative  at  San  Francisco,  who  in  turn  will  report 
directly  to  the  Director  for  this  project.  This  concept  will 
provide  the  project  office  access  to  the  Civilian  Personnel 
Center  and  the  DA  staff  through  the  Field  Representative.  The 
Field  Rep,  simply  by  the  nature  of  the  job,  has  frequent  contact 
with  field  operations  as  well  as  staff,  and  thus  can  easily 
oversee  the  scope  of  this  project  with  both  the  policy  levels  and 
the  operations  level  due  to  this  perspective  on  an  on-going 
basis . 

The  project  office  must  be  located  at  the  installation  at  which 
the  demonstration  project  will  be  undertaken.  This  gives  the 
project  office  direct  access  to  employee  records,  the  jobs, 
employees  and  supervisors  that  will  be  serviced  by  the  system, 
and  the  ADP  environment  in  which  the  system  will  operate. 

The  project  office  must  be  staffed  with  a  chief,  two  team 
members,  and  one  clerical  support  person.  The  chief  and  the  team 
members  must  be  qualified  personnel  management  specialists  with 
broad  backgrounds.  They  must  also  be  conversant  with  ADP  uses. 
Ideally,  the  team  members  should  also  be  knowledgeable  of  expert 
systems  technology,  however  based  on  my  research,  this 
combination  of  skills  cannot  be  realistically  assembled.  The  team 
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members  must  have  a  thorough  background  in  position 
classification.  Cost  estimates  of  this  project  are  based  on  the 
following  staffing  assumptions: 

1  Chief,  GM-14 

2  Team  Members,  GS-13 

1  Clerical  Support,  GS-6 

The  role  of  the  chief  is  to  provide  overall  direction  to  the 
project,  plan  activities  of  the  team  members,  serve  as 
contracting  officer’s  representative  overseeing  the  work  of  the 
contractor  in  the  development  of  the  expert  system  software, 
obtaining  resources  to  conduct  the  demonstration  project, 
coordinating  the  project  with  the  operating  CPO  and  activity 
Commander  when  it  is  brought  into  operation,  and  coordinating  the 
project  with  the  Field  Rep,  the  DA  staff,  OPM  and  other 
interested  parties.  The  role  of  the  team  members  are  to  oversee 
specific  aspects  of  the  project  such  as  the  development  of  the 
job  classification  module,  the  position  management  module,  and 
other  modules  required  so  that  the  system  will  mimic  a  human 
position  classifier.  Key  tasks  are  to  serve  as  the  initial 
expert  model  for  the  contractor  in  designing  the  system, 
coordinating  with  other  Army  experts  who  will  be  used  by  the 
contractor  to  purify  the  system’s  performance,  working  out 
details  of  how  different  regulatory  aspects  will  be  managed  by 
the  system,  and  coordinating  with  DA  and  CIVPERCEN  staff  members 
on  various  policies  which  will  be  handled  by  the  system.  The 
clerical  support  person  will  provide  typing,  filing,  telephone, 
and  office  administrative  support. 

Preliminary,  estimates  on  hardware/software  resources  required  are 
as  follows: 

One  mini-computer  with  six  workstations  (one  for  each  team  member 
and  the  clerk,  and  two  for  contractor  personnel).  It  is 
estimated  the  computer  must  be  a  Symbolics,  LMI  or  equivalent 
machine  for  programming  needs  by  the  contractor,  and  subsequent 
on-line  operation  of  the  expert  system.  A  computer  of  this  type 
runs  in  the  neighborhood  of  $150K.  Software  required  will  be 
driven  by  the  contractor.  A  tool  such  as  KEE  or  ART  costs  around 
$60K  to  $100K. 

Hardware/software  final  decisions  will  be  driven,  in  part,  by  the 
knowledge  engineer  consulting  firm  who  will  actually  develop  the 
system.  Preliminary  contacts  with  companies  capable  of 
supporting  this  project  resulted  in  the  above  estimate. 

An  office  of  at  least  1500  square  feet  to  house  the  project 
office  personnel,  contractor  personnel,  the  computer  and 
peripherals  is  also  required  at  the  installation  hosting  the 
demonstration  project.  The  office  must  be  equipped  with  desks, 
chairs,  filing  cabinets,  etc. 

The  contract  should  also  provide  for  contractor  training  of  the 
team  members  in  expert  systems  development  skills  and  techniques. 
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Thus,  as  the  contractor  develops  the  system,  the  team  members 
will  become  increasingly  capable  of  expert  systems  development  on 
their  own.  This  will  insure  an  ability  to  perform  maintenance 
during  the  operational  phase  of  the  demonstration  project. 


Alternative  #2. 

Under  this  alternative,  the  prototype  system  would  be  built  in- 
house  with  government  personnel.  A  project  office  team  of  seven 
persons  would  be  required  (details  covered  in  following  chapter), 
equipment  costs  would  be  essentially  the  same  as  above,  and  time 
to  develop  would  take  two  to  three  years.  A  field  location  for 
the  team  would  be  essential  for  the  same  reasons  as  cited  above, 
and  the  three  potential  sites  remain  the  same. 


My  recommendation  is  to  develop  the  prototype  with  in-house 
personnel.  There  are  drawbacks  to  this  recommendation,  but  1 
feel  the  benefits  outweigh  them.  Pros  and  cons  for  contractor 
vs.  in-house  are  as  follows: 

With  a  contractor,  the  Army  would  be  reasonably  assured  of  having 
a  professional  development  (from  a  computer  technical  point  of 
view)  of  the  prototype.  There  are  contractors  (at  least  a  few), 
who  have  experience  (albeit  little)  in  the  AI  techniques 
necessary  for  the  project.  Utilizing  in-house  personnel  will  not 
assure  that  professionals  at  expert  systems  development  will  be 
involved.  Personnel  will  have  to  be  trained  (covered  in  next 
chapter),  and  the  development  process  will  not  proceed  directly 
and  smoothly  from  start  to  finish.  Therefore,  a  more  rapid 
prototype  development  with  guaranteed  results  will  probably  be 
the  result  using  contractor  personnel,  while  a  more  methodical- 
while-learning  development  cycle  will  occur  with  in-house 
personnel.  There  are  some  risks  that  the  product  from  in-house 
personnel  may  not  be  as  efficient,  nor  as  professional  (code¬ 
wise)  as  the  contractor  product. 

The  contractor  costs  would  be  initially  high,  visible  and  known 
as  the  result  of  competitive  procurement.  In-house  costs  would 
be  somewhat  less,  initially,  not  as  visible  (mostly  salaries), 
and  not  totally  known,  since  time  factors  cannot  be  completely 
controlled  with  personnel  who  are  learning  as  they  go. 

In-house  personnel  would  have  a  commitment  to  the  project  that  is 
not  as  heavily  influenced  by  a  profit  motive  as  the  contractor. 
In-house  personnel  would  provide  an  autonomous  structure 
knowledgeable  of  the  domain,  and  increasingly  knowledgeable  of 
knowledge  engineering,  while  the  contractor  would  have  little 
commitment  to  become  any  more  proficient  in  the  knowledge  domain 
than  is  required  to  build  the  system. 

In-house  personnel  would  provide  a  continuing  resource  to 
maintain,  expand  and  improve  upon  the  system  at  no  further  cost 
other  than  salaries.  In  contrast,  a  contractor  developed  system 
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■ay  cause  an  "apron  string"  syndrome  to  be  developed  in  which 
continuing  reliance  on  the  contractor  for  systea  maintenance, 
expansion  and  improvement  would  result. 

Perhaps  most  important,  since  the  system  may  well  replace 
knowledge  workers  in  civilian  personnel  functions,  an  in-house 
development  effort  would  provide  an  opportunity  to  begin  the 
retraining  of  obsoleted  personnel.  The  "endangered  species"  (to 
quote  a  high  placed  government  official)  will  have  an  opportunity 
to  shift  with  the  changing  technology  to  another  field  of  work. 

My  recommendation,  therefore,  is  to  proceed  with  development  of 
the  prototype  with  in-house  personnel.  The  following  chapters  of 
this  report  cover  this  proposal  in  more  detail. 
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In  this  chapter  I  will  define  a  recommended  project  office  to 
include  its  size,  organizational  location,  qualifications  of 
personnel  that  will  be  required,  and  training  needs  for  these 
personnel.  The  project  office  that  will  be  described  is  based  on 
the  assumption  that  the  prototype  system  will  be  developed  with 
in-house  -personnel. 

Qualifications  of  project  office  personnel. 

Personnel  assigned  to  the  expert  system  project  office  must  have 
unique  qualifications,  or  the  ability  to  assimilate  the 
qualifications  in  a  very  short  period  of  time.  Ideally,  these 
personnel  would  be  qualified  as  both  personnel  generalists  in  the 
200  series,  with  extensive  backgrounds  in  position  classification 
and  computer  programmers  in  the  334  series.  Further,  the 
qualifications  in  the  334  series  should  be  in  AI  techniques  with 
skills  in  programming  in  LISP.  This  ideal  situation  is  not 
attainable  immediately,  since  personnelists  are  not  skilled  in 
the  334  series,  and  programmers  are  not  available  with  the 
requisite  personnel  experience  and  qualifications.  It  will  also 
be  difficult  to  ,find  334  personnel  with  AI  experience. 

m 

Thus,  it  will  be  necessary  to  teach  new  skills  and  abilities  to 
either  personnelists  or  programmers.  An  alternate  approach  would 
be  to  assemble  a  team  consisting  of  specialists  from  both 
disciplines.  This  latter  alternative  will  be  explored  first. 

To  build  a  team  consisting  of  some  personnelists  and  some 
programmers  and  expect  each  disciplinary  function  to  remain  pure, 
i.e.  the  personnelists  never  to  touch  the  programming  environment 
that  is  the  exclusive  domain  of  the  programmers,  and  the 
programmers  to  never  deal  substantively  with  the  domain  of 
knowledge  in  which  they  are  writing  code,  would  not  work  well. 
Turf  battles  would  inevitably  ensue,  and  excuses  for  delays  or 
problems  would  always  be  laid  at  the  feet  of  the  other  group. 
Such  a  concept  for  the  project  team  would  insure  the  longest 
development  time  for  the  project  to  become  operational.  I  would 
strongly  recommend  against  this  approach.  For  those  who  would 
reject  this  recommendation  out  of  hand  based  on  the  idea  that 
specialists  are  required  for  each  specialized  task,  I  would  ask 
them  to  look  closely  at  the  SCIPMIS  system,  and  how  long  it  has 
taken  to  get  it  to  the  archaic  state  that  it  currently  exhibits. 

To  teach  programmers  to  become  personnelists  is  almost  too 
incongruous  to  mention.  While  this  might  be  remotely  possible, 
givon  a  lot  of  time,  the  credibility  of  the  product  produced  by 
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retrained  programmers  would  be  nil. 

Training  personnelists  to  be  programmers  may  at  first  seem  to  be 
a  long  process  with  a  great  deal  of  risk  involved  that  these 
basically  liberal  arts  personnel  could  ever  learn  the  difficult 
science  of  computer  programming.  But  the  idea  is  not  as  far 
fetched  as  it  might  first  appear.  Evidence  the  microcomputer 
revolution  that  has  occurred  in  this  country  since  the  mid¬ 
seventies.  Many  of  these  people  involved  in  this  phenomenon  are 
not  scientifically  oriented  people.  They  are  basically  people 
with  inquisitive  minds  who  begin  to  poke  around  with  a 
microcomputer,  read  as  much  as  they  can  get  their  hands  on,  and 
teach  themselves  the  rest.  Many  have  taught  themselves  how  to 
program  not  only  in  one  of  the  higher  level  languages  such  as 
BASIC  and  Pascal,  -but  also  the  lQwer  level,  and  much  more 
difficult  languages  such  as  assembly.  Learning  a  programming 
language  is  not  all  that  difficult. 

LISP  is  one  of  the  more  difficult  higher  level  languages  to 
learn.  However,  it  is  the  programming  language  of  choice  of 
computer  scientists  for  AI  programming,  and  most  expert  systems 
have  been  programmed  in  this  language.  This  is  no  accident. 
LISP  is  idea-lly  suited  to  expert  systems,  while  many  of  the  more 
traditional  languages  such  as  FORTRAN  are  not.  LISP  is  an  object 
oriented  language.  Expert  systems  can  best  be  developed  with 
this  type  of  language. 

How  would  we  train  personnelists  in  LISP  programming?  First  of 
all,  I  do  not  believe  that  the  team  members  need  to  be  expert 
LISP  programmers.  Elsewhere  in  this  report  recommendations  are 
made  for  hardware  and  software  to  support  the  project  office. 
The  expert  system  development  tool  recommended  takes  much  of  the 
programming  requirements  out  of  process.  Some  LISP  programming 
is  required  to  supplement  the  tool.  Writing  small  segments  of 
LISP  code  would  be  required  to  use  this  tool,  while  the  tool 
would  take  care  of  most  of  the  tedium  of  developing  the  expert 
system.  In  other  words,  the  entire  expert  system  would  not  have 
to  be  LISP  coded  from  scratch. 

I  believe  that  any  member  of  the  team  must  be  willing  to  learn 
and  have  the  capacity  to  program  in  LISP  in  order  to  be  both 
productive  as  a  personnelist  on  the  team,  and  to  be  a  hands-on 
worker  in  the  development  of  the  system  software.  This  will  best 
insure  a  product  that  is  both  professionally  done  from  a 
personnel  point  of  view,  and  is  a  computer  program  that  does  what 
it  is  supposed  to  do.  Thus,  personnelists  must  be  selected  for 
the  project  team  who  meet  these  criteria. 

The  Department  of  Navy  has  funded  a  research  project  at  Carnegie- 
Mellon  University  that  resulted  in  a  program  known  as  the  LISP 
Tutor.  This  program  does  a  very  credible  job  of  teaching  new 
students  to  program  in  LISP.  CMU’s  experience  has  shown  that 
students  learning  LISP  in  the  conventional  classroom  environment 
do  not  learn  as  efficiently,  nor  as  fast  as  those  with  human 
tutors  to  help  them.  Thus  the  LISP  Tutor  program  was  developed 


46 


to  replace  the  human  tutor.  This  program  has  been  shown  to  be 
nearly  as  efficient  at  teaching  LISP  as  human  tutors  are,  and 
much  more  effective  at  teaching  students  than  traditional 
classroom  instruction. 

I  contacted  the  Office  of  Naval  Research,  and  have  found  that  the 
LISP  Tutor  could  be  made  available  to  the  Department  of  Army.  At 
the  time  of  my  contact,  I  was  informed  that  CMU  had  not  finished 
the  documentation  for  the  program,  but  the  contract  did  allow  the 
Government  to  use  the  program  without  royalties,  or  some  small 
fee  at  the  maximum.  The  final  product  was  not  expected  from  CMU 
until  late  )une  or  July.  I  recommend  Army  further  pursue 
obtaining  the  LISP  Tutor  for  use  in  training  personnel ists 
assigned  to  the  project  office  for  the  prototype  system  if  this 
method  is  chosen  to  develop  an  expert  system.  This  program  will 
run  on  any  VAX  computer  that  runs  under  the  VMS  operating  system. 

I  also  recommend  that  the  project  office  have  one  person  assigned 
whose  background  is  purely  computer  science.  This  person  should 
be  an  accomplished  LISP  programmer,  or  one  who  is  familiar  with  a 
number  of  computer  languages  and  could  quickly  learn  to  program 
in  LISP.  This  single  programming  specialist  is  required  since 
one  true  expert  in  programming  is  needed  to  be  available  to  the 
team  as  difficulties  arise,  as  they  most  certainly  will.  With 
only  one  334-type  on  the  team,  the  problems  associated  with 
specialist  "camps"  will  not  occur.  It  will  probably  be  difficult 
to  find  an  experienced  LISP  programmer  since  they  are  in  great 
demand.  If  one  cannot  be  found,  we  should  be  able  to  find  a  good 
334  with  the  interest  and  ability  to  learn  LISP  at  a  fairly  rapid 
pace. 

Size  and  composition  of  the  project  office. 

The  recommended  size  and  composition  of  the  team  for  the  project 
office  is  as  follows: 

1  Project  Office  Chief,  GM-201-14..  The  individual  must  have  a 
wide  background  in  personnel  administration,  ideally  having  held 
a  Personnel  Officer  job.  The  reason  for  this  experience  is  to 
insure  the  prototype  is  developed  with  equal  weight  being  placed 
on  all  functions  it  will  perform,  and  insure  integration  of  them 
into  a  cohesive  whole.  Expansion  possibilities  into  other 
functional  areas  in  the  future  must  receive  appropriate  attention 
as  this  prototype  is  built.  It  is  also  important  that  the  chief 
of  the  office  be  experienced  at  managing  a  function  with  a  strong 
view  toward  service.  The  person  should  be  adept  at  getting 
things  done  in  the  face  of  many  distractions,  and  perhaps  even  in 
spite  of  popular  opinion.  The  ability  to  organize,  plan  and 
evaluate  are  crucial  to  success.  The  successful  candidate  will 
require  some  background  in  ADP  activities,  and  be  committed  to 
bringing  about  an  operating  demonstration  project  that  satisfies 
all  interests  (policy  makers  as  well  as  serviced  employees  and 
supervisors).  This  person  needs  to  be  unafraid  to  roll  up 
sleeves  and  be  a  productive  member  of  the  team,  to  include 
working  directly  with  the  computer  to  develop  the  system. 
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2  Personnel  Specialists,  GS-201-13.  These  individuals  must  also 
have  broad  backgrounds  in  personnel,  ideally  in  at  least  two 
functional  areas.  A  strong  position  classification  specialist 
background  is  essential.  Since  others  will  be  brought  to  the 
team  from  time-to-time  to  serve  as  expert  models,  and  for  testing 
purposes,  these  people  need  to  be  willing  to  serve  as  the  initial 
experts  to  build  the  initial  system  in  specialized  areas,  but 
have  the  capacity  to  accept  changes  based  on  other  experts 
inputs.  As  the  system  is  "taught”  its  expertise,  these 
personnelists,  along  with  the  chief,  will  serve  as  the  designers 
of  that  expertise,  and  its  user  interface.  They  must  have  the 
ability  to  learn  to  program  in  LISP,  and  have  a  keen  interest  in 
computers  and  computer  applications  to  the  personnel  business. 

1  Computer  Programmer,  GS-334-13.  This  individual  will  ideally 
be  an  accomplished  LISP  programmer.  Failing  to  find  such  an 
individual,  the  person  must  be  a  highly  skilled  programmer  in 
other  languages  with  a  strong  desire  to  learn  LISP  programming 
quickly  and  be  a  productive  member  (a  LISP  guru)  on  the  team.  A 
spirit  of  team  cooperation  is  an  essential  personality  trait. 

1  Electronics  Technician,  GS-856-7/9/11/12.  This  individual  will 
serve  as  the  minion  to  the  hardware.  He/she  will  also  perform  a 
variety  of  other  general  technical  tasks,  and  must  exhibit  a  high 
degree  of  skill  in  keeping  computer  hardware  operational,  making 
connections  of  hardware  items  as  required,  and  serving  as  the 
hardware  interface  expert  for  the  system  as  it  is  integrated  on 
line  for  supervisor’s  use.  Some  programming  tasks  will  be 
performed  by  this  person. 

1  Administrative  Assistant,  GS--???-7/9.  This  individual  will 
perform  various  .  administrative  tasks  including:  budget 
preparation  and  administration;  editorial  assistance  on  reports, 
documentation,  user  manuals,  etc.;  briefing  materials  preparation 
(BASIC  or  Pascal  programming  will  be  required  for  this); 
preparation  of  specifications  for  procurement  actions;  and  other 
tasks  as  required. 

1  Secretary,  GS-318-6.  This  individual  will  serve  as  the  office 
administrator  and  provide  clerical  support  to  all  members  of  the 
team. 

While  the  grades  of  these  jobs  may  seem  high  at  first  glance,  it 
must  be  remembered  that  the  team  is  working  on  an  Army-level 
project  that  may  well  be  used  in  place  of  many  personnelists  in 
the  future.  Thus,  the  impact  of  the  work  accomplished  by  the 
team  is  potentially  agency-wide,  and  the  skills,  knowledges  and 
abilities  of  the  members  of  the  team  must  be  at  the  top  end  of 
the  spectrum  in  order  to  insure  a  successful  project  is  fielded. 


Organizational  location  of  project  office. 

There  are  a  multitude  of  possibilities  for  this.  The  ultimate 
decision  will  be  driven  by  such  things  as  sources  of  funding, 
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influences  of  powerful  people  and  the  like.  I  am  going  to  be  so 
bold  as  to  suggest  how  I  feel  it  should  be  placed.  This 
recommendation  was  previously  made  in  the  chapter  9. 

An  organizational  wiring  diagram  appears  in  Figure  4.  Because  of 
the  recommended  installations  at  which  the  prototype  should  be 
developed,  the  San  Francisco  Field  Office  makes  a  logical  choice 
for  developmental  oversight. 
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Figure  4:  Organizational  chart  depicting  the  recommended 

placement  of  the  project  office. 


A  following  chapter  will  give  additional  information  about 
possible  installations  at  which  to  locate  the  project  office. 
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In  order  to  build  the  prototype,  some  computer  equipment  will 
need  to  be  obtained.  Equipment  can  either  be  found  within  the 
current  resources,  leased,  or  purchased.  Equipment  will  consist 
of  computer  hardware  and  software.  I  accomplished  some 
preliminary  investigations  into  the  kinds,  costs  and  availability 
of  equipment  needed,  to  support  an  in-house  development  effort. 
This  was  accomplished  to  prepare  cost  estimates  for  the  prototype 
development.  If  Army  proceeds  to  pursue  this  effort,  I  recommend 
the  project  team  chief,  and  the  ADP  specialist  examine  equipment 
needs  in  considerable  detail  to  insure  the  right  equipment  is 
obtained  in  order  to  minimize  any  delay  proper  equipment  might 
have  on  the  prototype  building  endeavor.  Such  an  analysis  will 
probably  result  in  some  changes  to  these  estimates. 

Most  expert  systems  are  built  on  fairly  large  computers, 
mainframes  or  large  minis.  Expert  systems  demand  large  amounts 
of  memory  capacity,  both  random  access  and  disk  based.  Although 
"toy”  systems,  such  as  the  demonstration  module  I  built  on  the 
microcomputer  used  on  this  fellowship  can  be  built  on  this  kind 
of  machine,  serious  applications  demand  heavyweight  computing 
power.  Since  the  next  stage  of  development  is  the  prototype 
stage,  and  since  this  is  still  an  R&D  effort,  I  do  not  recommend 
Army  invest  in  a  large  mainframe  simply  to  build  a  prototype.  A 
full  working  system  in  a  standard  system  format  would  require  a 
mainframe  computer.  However,  at  this  stage  of  development,  a 
prototype  could  be  built  on  a  less  expensive  machine. 

The  machine  is  driven  somewhat  by  the  software  development  tool 
that  will  be  used  for  building  the  system.  Therefore,  I  will 
first  address  the  software  issue.  As  mentioned  in  previous 
chapters,  systems  can  be  coded  from  scratch  using  LISP  or  Prolog, 
or  some  other  language.  However,  I  wouldn’t  recommend  this 
approach  due  to  several  factors.  First,  programming  in  a 
language  is  very  slow  going.  The  prototype  needs  to  be  developed 
as  rapidly  as  possible.  Second,  finding  personnel,  or  training 
them  to  be  efficient  programmers  to  tackle  the  problem  in  this 
manner  would  be  a  very  risky  business,  and  consume  a  lot  of  time. 
And,  third,  system  maintenance  and  expansion  would  be  more 
laborious  if  a  language  were  used. 

There  are  tools  available  or  the  market  which  help  to  speed 
development  time  of  systems  and  are  easier  to  learn  to  use  than  a 
language.  Some  of  them  require  developers  to  write  some  code, 
and  others  require  no  programming  effort.  I  have  looked  at  KEE 
from  IntelliCorp,  and  ART  from  Inference  Corp.  These  are  very 
powerful  development  tools  capable  of  handling  the  proposed 
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prototype.  I  have  read  a  considerable  amount  about  other  tools 
such  as  BMYCIN,  LOOPS,  0PS5 ,  KBS,  S.l,  TIMM  and  others.  They  are 
possible  candidates  for  this  project  also.  Since  ay  purpose  on 
this  project  was  not  to  select  the  tool  to  be  used,  I  will  not 
recoaaend  one.  Additional  research  by  the  development  teaa  would 
finalize,  the  selection. 

A  dollar  figure  of  about  $100K  has  been  estimated  for  the  tool. 
This  is  will  procure  the  software,  a  support  package,  and  some 
training  in  the  tool's  use  by  the  manufacturer. 

These  tools  are  implemented  on  various  hardware  systems.  The 
powerful  and  recommended  tools  run  on  LISP  machines,  DEC-VAX 
machines,  and  others.  The  costs  of  these  machines  varies,  of 
course,  but  I  would  estimate  that  a  procurement  cost  of  about 
$150K  to  $200K  would  be  required  for  the  main  hardware. 

I  would  also  recommend  investment  in  some  other  equipment  to 
support  the  team  in  this  phase  of  development.  Each  team  member 
should  have  a  microcomputer  equivalent  to  the  one  used  on  my 
research  project.  These  systems  are  very  productive  tools  for 
the  myriad  of  peripheral  tasks  that  will  be  associated  with  the 
actual  prototype  development.  As  a  matter  of  course,  I  found 
that  I  spent  approximately  70*  to  80*  of  my  time  at  the  computer 
on  this  project.  This  is  in  addition  to  the  effort  to  develop 
the  demo  module  in  which  100*  of  my  time  was  at  the  computer. 
Writing,  conducting  cost  analysis  with  spreadsheets,  maintaining 
records  with  the  database  manager,  preparing  briefings,  and  many, 
many  tasks  are  performed  much  more  efficiently  and  quickly  using 
personal  computers.  They  are  clearly  .the  "tool  of  the  knowledge 
worker".  Each  system  could  be  procured  for  about  $10K  to  $15K 
and  are  strongly  recommended.  With  seven  people  on  the  team,  the 
total  cost  would  be  in  the  neighborhood  of  $70K  to  $100K. 

A  rough  estimate  of  $300K  to  $400K  total  start-up  costs  for  ADP 
equipment  is  envisioned. 
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An  installation  will  need  to  be  selected  at  which  to  develop  and 
test  the  prototype  expert  system.  During  the  course  of  my 
project,  I  made  some  preliminary  evaluations  of  possible  sites. 
An  ideal  test  site  would  have  several  features.  It  would  have  a 
very  extensive  distributed  computer  network  already  in  place  that 
could  have  this  system  integrated  on  line  with  existing  systems. 
All  supervisors  would  have  access  to  the  network,  and  all 
current  standard  systems  (SCPIMIS,  STARCIPS,  budget,  etc.)  would 
be  integrated  into  the  network. 

The  ideal  installation  would  also  have  employed  a  workforce  of 
employees  and  supervisors  who  had  a  high  degree  of  computer 
literacy,  and  had  an  understanding  of  expert  systems  and  a  belief 
in  their  ability  to  mimic  human  reasoning  capacity.  The 
personnel  office  personnel  at  the  test  installation  would  have  an 
enthusiasm  for  automating  decision  processes,  and  would  be 
knowledgeable  of  traditional  computer  uses.  An  ability  to  accept 
the  basic  tenants  of  R8.D  projects  which  cannot  be  totally  defined 
at  the  outset,  and  which  suffer  many  setbacks  as  the  research  and 
development  progresses  would  be  prevalent  in  attitudes  of  the 
installation  personnel  at  all  levels. 

The  geographic  location  of  the  installation  would  be  generally 
desirable  by  people  from  outside  the  area.  By  this  I  mean  that 
it  would  not  have  serious  shortcomings  or  undesirable  attributes 
that  would  cause  potential  employees  of  the  project  office  to  shy 
away  from  relocating  there.  Very  high  costs  of  living,  extreme 
climatic  conditions,  remoteness  from  cultural,  educational, 
social  and  recreational  benefits  are  some  aspects  that  should  be 
avoided. 

The  population  of  the  installation  should  be  large  enough  so  that 
one  or  two  of  its  organizational  entities  could  be  used  as  the 
test  organization  without  causing  undue  disruption  to  other 
activities  at  the  installation.  I  do  not  anticipate  the  tests 
will  cause  disruption,  but  if  a  very  tiny  installation  were 
selected,  and  the  only  organization  of  sufficient  size  to  serve 
as  the  test  was  the  major  mission  element  of  the  activity,  then 
potential  problems  could  arise  that  might  be  construed  as 
disruption . 

I  do  not  believe  there  are  any  installations  within  Army  that 
satisfy  all  of  the  above  "ideal"  conditions. 

Three  installations  have  been  identified  that  may  come  closest  to 
these  parameters.  They  are  White  Sands  Missile  Range,  Sacramento 
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Aray  Depot,  and  one  or  more  of  the  District  Offices  in  the  North 
Pacific  Division,  COE.  Personnel  office  personnel  at  each  of 
these  activities  coae  fairly  close  to  criteria  specified  above. 
There  seems  to  be  enthusiasm,  and  acceptance  of  automation 
efforts,  at  least  in  my  preliminary  assessment.  Each  are  of 
sufficient  size  to  accommodate  a  non-disruptive  test.  Each  are 
located  in  desirable  places  (WSMR  may  be  the  least  of  the  three). 

As  far  as  good  networked  automation  efforts  are  concerned,  there 
are  no  Army  activities  that  have  desirable  environments  with  the 
possible  exception  of  my  home  installation,  Yuma  Proving  Ground. 
However,  Yuma  is  a  bit  small,  and  it  is  definitely  not  in  a 
desirable  geographical  location,  except  for  those  who  have  lived 
there  for  awhile.  Convincing  others  of  the  attractive  benefits 
of  living  in  the  desert  climate  of  Yuma  in  a  fairly  small  city 
would  be  a  very  difficult  task  in  recruitment  efforts.  I  know, 
our  personnel  office  recruiters  face  this  problem  on  a  daily 
basis.  Otherwise,  Yuma  has  made  great  strides  on  an  installation 
wide  basis  at  automation  and  distributed  systems.  It  does  not 
meet  the  ideal  outlined  above,  but  it  might  well  be  the  best  of 
the  lot. 

In  my  estimation,  WSMR,  NPD  or  Sacramento  are  the  viable  choices 
for  hosting  the  development  of  the  prototype,  and  I  recommend  the 
selection  be  from  among  these  three,  unless  further  analysis 
identifies  an  installation  that  more  nearly  matches  the  "ideal". 
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In  this  chapter  I  will  outline  an  estimated  time  schedule  for 
development  of  the  PM&C  prototype  system.  A  basic  assumption  made 
is  the  system  will  be  developed  in-house.  If  the  prototype  were 
developed  under  contract,  one  might  expect  the  times  to  be  less 
since  there  are  no  time  requirements  for  training  the  contractor 
personnel  in  knowledge  engineering  activities.  Once  the 
contractor  understood  the  problem,  he  would  be  equipped  to  apply 
known  knowledge  engineering  techniques  to  the  problem  at  a  rather 
rapid  pace. 

These  estimates  are  gross  approximations.  Without  having  used 
the  tools  that  will  be  required  to  develop  the  prototype,  I  can 
only  guess  at  certain  time-frames.  The  outline  for  this  schedule 
can  serve  to  estimate  costs,  and  to  give  a  development  team  some 
goals  to  shoot  for. 

Figure  5  shows  an  estimated  milestone  schedule.  A  task  breakout 
of  some  specific  milestones  is  included  in  Appendix  C. 
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Figure  5:  Milestones  for  prototype  development. 


During  start-up,  which  would  take  from  six  to  nine  months,  the 
project  office  would  be  staffed,  equipment  would  be  procured,  and 
training  on  the  hardware,  software  and  knowledge  engineering 
would  begin. 


During  the  next  12  to  18  months,  the  team  would  build  the 
prototype  system.  In  this  stage,  the  team  serves  as  both  the 
expert  and  the  knowledge  engineer.  This  is  the  most  critical 


period  of  the  project. 

The  validation  stage,  which  would  take  six  to  nine  months,  would 
be  the  period  in  which  other  Army  experts  are  utilized  to  fine- 
tune  the  prototype  for  use  in  the  test  organization.  Experts 
would  be  borrowed  from  throughout  Army  to  assess  the  reasoning 
ability  of  the  prototype,  and  provide  their  expertise  in  assuring 
the  system  is  drawing  correct  conclusions. 

The  next  12  months  would  be  used  in  actually  testing  the  system 
in  operation  in  the  test  organization.  Supervisors  would  use  the 
system  directly  for  the  functions  it  had  been  developed  to 
perform. 

During  this  same  period,  evaluation  of  the  system  would  be 
conducted  by  having  parallel  and  identical  servicing  by  the 
operating  personnel  office,  observation  of  its  performance  by  the 
team  and  Army  Staff  personnel,  etc.  (see  following  chapter  on 
evaluation  of  expert  systems).  The  evaluation  at  the  conclusion 
of  the  year  test  phase  would  take  about  three  months,  and  would 
deal  with  issues  such  as  probable  cost  savings,  overall 
performance  of  the  prototype,  areas  for  improvement,  expansion, 
etc . 
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Because  of  the  relative  newness  of  this  technology,  expert 
systeas  in  use  in  real  world  settings  are  not  plentiful.  As  a 
consequence,  there  is  not  a  great  deal  of  information  available 
to  assess  the  practical  benefits  of  their  usage.  It  has  been 
recognized  by  some  that  the  evaluation  of  expert  systems  is  vital 
to  researchers,  funders  of  projects,  and  the  using  world  (both 
government  and  private  companies)  for  the  simple  reason  that 
expert  systems,  if  they  are  viable  at  all,  should  be  able  to 
prove  their  worth.  Those  who  have  recognized  the  importance  of 
evaluation  also  stress  the  importance  of  planning  for  the 
evaluation  process  early  on,  rather  than  wait  for  the  project  to 
be  put  into  operation,  and  then  decide  how  it  will  be  evaluated. 

Evaluations  have  been  accomplished  on  some  of  the  systems  that 
are  currently  employed  in  a  using  environment.  For  example,  Rl, 
the  Digital  Equipment  Corp.  expert  system  used  to  configure  VAX 
computers  has  been  extensively  evaluated.  These  evaluations  have 
assessed  not  only  the  performance  of  how  well  it  performs  its 
task,  but  also  other  factors.  On  the  other  hand,  systems  such  as 
MYCIN  have  been  primarily  evaluated  only  on  quality  of  the 
decisions  and  advice  rendered.  Other  factors  such  as  the 
correctness  of  reasoning,  the  quality  of  the  system/user 
interaction,  efficiency  and  cost  effectiveness  have  not  been 
assessed  in  much  detail,  if  at  all. 

This  section  of  the  report  will  be  concerned  with  the  critical 
issue  of  evaluation.  Should  further  pursuit  of  applying  expert 
systems  technology  to  personnel  administration  be  undertaken, 
this  will  outline  a  method  of  evaluation. 

The  several  stages  of  implementing  expert  systems  has  been 
discussed  in  chapter  3  of  this  report.  From  the  outset,  when  the 
system  is  initially  designed,  evaluation  must  be  made  a  part  of 
the  top-level  design.  This  is  the  stage  at  which  the  definition 
of  long  range  goals  are  established.  At  the  same  time  explicit 
statements  of  what  the  measures  of  the  program’s  success  will  be 
and  how  that  failure  or  success  will  be  evaluated  must  be  made. 
Based  upon  the  recommendations  made  in  this  report  to  design  a 
system  to  accomplish  PM&C  functions,  the  statements  associated 
with  this  goal  will  now  be  proposed. 

1.  Accuracy.  The  system  must  provide  a  level  of  accuracy 
in  its  decisions  and  recommendations  at  least  as  accurate  as 
those  that  would  be  rendered  by  human  personnelists .  This  will 
be  measured  in  several  ways. 
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a.  First,  the  average  level  of  accuracy  of  decisions 
rendered  by  human  personnelists  will  be  measured.  For  example, 
the  usual  error  rate  of  classifying  jobs  by  personnelists  in  the 
field  will  serve  as  one  measure.  Their  accuracy  rate  must  be 
determined  by  a  panel  of  experts.  It  is  recommended  Army  level 
experts  be  used  to  assess  the  personnelists  level  of-  accuracy  as 
well  as  the  system's.  (It  was  found  in  tests  of  MYCIN  that  it 
performed  at  an  accuracy  rate  of  about  75%,  which  at  first  was 
thought  to  be  disappointing  to  the  researchers  who  were  expecting 
90%.  When  diagnoses  of  diseases  by  experts  were  gathered,  it  was 
found  that  even  they  function  at  only  a  70-80%  accuracy  rate.) 
One  form  of  evaluation  of  accuracy  under  this  scheme  must  be  a 
blind  type  of  test.  For  example,  the  panel  of  experts  should  not 
have  knowledge  of  whose  judgments  they  are  assessing, 
personnelists  or  the  expert  system.  In  this  manner,  biases  are 
eliminated. 

b.  Second,  a  so  called  gold  standard  will  be  used  in 
those  areas  where  it  fits.  A  gold  standard  is  one  in  which  a 
final,  objective  result  is  measurable.  For  example,  a  gold 
standard  exists  for  such  a  decision  in  personnel  as  the  number  of 
hours  of  annual  leave  an  employee  will  accrue.  It  is  based  on 
SCD,  and  once  calculated  there  is  no  question  as  to  its  accuracy, 
unless  the  mathematical  calculation  was  erroneous.  No  subjective 
measures  enter  in  as  they  do  with  judgments  about  the  grade  of  a 
job.  In  the  MYCIN  project,  a  gold  standard  existed  based  on 
autopsies,  and  other  objective  measurements  in  some  cases.  To 
the  extent  that  gold  standards  pertain  to  the  prototype,  they 
should  be  i3>d. 

c.  Third,  progressive  tests  to  eliminate  the 
possibility  of  a  lack  of  accuracy  being  attributable  to  either 
non-acceptance  by  users,  or  changes  in  the  system  affecting 
previous  abilities.  What  this  means  is,  as  the  system  is 
developed,  experts  being  used  to  develop  it  must  be  convinced  of 
its  accuracy  before  placing  the  system  into  operation.  Thus,  if 
judgments  rendered  by  the  system  are  rejected  as  inaccurate  by 
users,  the  inaccuracy  can  more  readily  be  attributed  to  the 
user's  perception,  rather  than  a  fallacy  in  the  system.  It  also 
means  that  a  library  of  cases  needs  to  be  maintained  so  that  they 
can  be  batch  processed  through  the  system  after  changes  are  made 
to  it.  Changes  to  improve  on  one  performance  area  in  such 
complex  systems,  may  cause  degradation  in  other  areas.  The 
library  cases  can  be  utilized  to  test  for  degradations  after 
changes  are  made. 

2.  Reasoning  techniques.  A  methodology  needs  to  be 
developed  to  examine  the  correctness  of  the  reasoning  techniques 
used  by  the  expert  system.  It  will  not  be  acceptable  if  the 
system  simply  provides  correct  answers.  The  manner  in  which  the 
system  reached  these  conclusions  will  also  need  to  be  clearly 
evident,  and  this  manner  subjected  to  the  scrutiny  of  experts  for 
evaluation.  For  example,  in  both  MYCIN  and  PROSPECTOR  this  kind 
of  evaluation  was  not  performed.  If  the  system  provided  what  was 
judged  to  be  quality  expert  advice,  the  reasoning  processes  used 


by  the  system  were  not  questioned.  Because  there  are  so  many 
"experts"  in  our  business  of  civilian  personnel  administration, 
an  ability  to  evaluate  the  techniques  used  by  a  system  to  infer  a 
conclusion  is  critical  to  its  success. 

3.  Human-computer  interaction.  Because  of  the  public 
environment  in  which  the  system  is  targeted  to  operate, 
evaluation  of  this  interface  is  very  important.  Careful  and 
intensive  planning  must  be  brought  to  bear  on  the  design  of  this 
interface,  and  testing  and  evaluation  of  it  throughout  the 
development  of  the  system  must  be  insured.  It  may  seem  difficult 
to  formally  evaluate  this  interface  during  the  development  phase, 
since  the  true  test  will  not  occur  until  the  system  is  put  into 
operation,  and  users  begin  to  provide  feedback.  However,  if  the 
development  phase  is  not  conducted  in  the  laboratory  setting,  but 
at  the  site  where  the  system  will  be  tested  in  an  operational 
mode,  the  potential  users  can  be  brought  into  the  development 
processes  early  on.  Such  important  factors  to  consider  in  human- 
computer  interaction  are: 

a.  The  choice  and  structure  of  words  and  phrases  used 
in  the  questions  and  responses  generated  by  the  system. 

b.  The  ability  of  the  system  to  explain  itself  to  the 
user  as  to  why  it  needs  certain  information,  or  why  a  certain 
conclusion  was  reached.  As  a  fielded  system  (paes)  will 
certainly  be  used  by  a  wide  variety  of  people  with  varying 
degrees  of  knowledge  of  personnel  administration. 

c.  To  quote  a  much  over-used  phrase,  "user 
friendliness",  or  the  ability  of  the  system  to  overcome  users  who 
may  fear  or  distrust  computers,  especially  in  this  field  of 
expertise.  This  includes  the  ability  of  the  system  to  assist 
users  in  the  operation  of  the  system  in  an  interactive  mode. 

Bringing  supervisors  and  employees  into  the  project  early-on  as 
it  is  being  developed  will  reap  untold  benefits  in  the  changes 
that  are  made  to  insure  a  good  human-computer  interface,  and  will 
help  to  prepare  the  test  users  for  actual  use  of  the  system. 

4.  System  efficiency.  The  system  will  need  to  be  at  least 
as  efficient  as  its  human  counterparts.  In  fact,  a  design  goal 
should  be  a  system  several  orders  of  magnitude  greater  than 
humans.  Efficiency  can  be  evaluated  from  two  broad  perspectives. 

a.  First  is  the  perspective  of  technical  efficiency  as 
a  computer  program.  This  is  an  analysis  of  the  program  in  terms 
of  its  use  of  CPU  (central  processing  unit)  time,  its  memory 
management  efficiencies,  its  disk  access  non  productive  time,  its 
judicious  allocation  of  time  to  multiple  users,  its  tight  and 
effective  code,  etc.  In  this  area  of  efficiency  the  design  goals 
are  that  the  system  operate  from  one  machine  to  service  multiple 
users,  that  no  obviously  perceptible  delays  are  observed  by  the 
user  unless  explained  by  the  system  to  the  user,  that  optimum 
utilization  is  made  of  the  cpu,  memory  and  disk  storage  media, 
and  that  system  down  time  due  to  regular  maintenance  (backup, 
etc.)  be  no  more  than  3%  of  a  16  hour  day. 
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b.  Second  is  the  perspective  of  user  efficiency.  I  do 
not  believe  it  is  unreasonable  to  pose  a  design  goal  for  this 
systea  in  the  area  of  job  classification  of  75  to  90  percent 
decrease  in  tiae  to  classify  a  job  froa  the  start  of  the  process 
of  a  supervisor  desiring  to  have  a  job  classified  until  the  job 
is  completed. 

An  evaluation  strategy  needs  to  be  determined  early  on  in  the 
development  phase  of  the  prototype  lest  the  system  be  brought  to 
use  with  no  means  of  examining  its  worth. 
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Expert  systeis  cannot  be  built  without  a  human  expert  whose 
knowledge,  reasoning  strategies  and  abilities  are  aodeled  in  the 
expert  system's  inference  engine  and  knowledge  base.  Development 
of  all  expert  systeas  has  used  the  approach  of  the  knowledge 
engineer  working  with  a  domain  expert  to  capture  the  human’ s 
expertise.  In  soae  recent  cases,  using  software  development 
tools,  the  expert  hiaself  has  built  the  expert  system.  However, 
in  all  cases  the  huaan  expert  is  essential,  and  as  Feigenbaua  has 
stated,  nearly  sufficient,  to  build  the  expert  system.  During 
the  course  of  this  research  project,  I  undertook  the  task  to 
locate  Aray’s  experts  in  the  various  specialized  areas  in 
personnel  administration.  I  was  assisted  in  this  effort  by  Mr. 
Severin  Johnson,  CPO,  Oakland  Aray  Base,  who  Bade  aany  of  the 
follow-up  contacts  in  search  of  Aray’s  experts. 

The  method  used  to  identify  experts  was  to  first  make  contact 
with  Civilian  Personnel  Officers  and  Directors  at  all  CONUS 
CPO's.  Contact  was  not  established  with  OCONUS  CPO’s  due  to  the 
methodology  which  would  be  used,  naaely  telephonic  interviews 
with  personnel  professionals  in  seeking  to  identify  experts.  In 
this  first  contact,  the  purpose  was  specified,  and  the  term 
"expert"  was  defined  as  "one  with  special  knowledge  representing 
aastery  of  a  particular  subject.  Bxperts  are  also  those  to  whom 
other  specialists  in  the  subject  aatter  area  go  to  seek  advice, 
counsel,  and  solutions."  If  one  were  to  try  and  identify  experts 
in  a  purely  objective  Banner,  soae  fora  of  test  would  have  to  be 
devised,  validated,  and  administered  to  discover  those  who  have  a 
aastery  of  a  subject.  However,  since  experts  are  those  to  whoa 
others  seek  counsel  and  solutions  to  difficult  problems,  it 
seeaed  adequate  to  accomplish  the  identification  process  through 
the  method  chosen. 

As  each  personnel  officer  was  contacted,  questions  were  posed 
about  whoa  the  CPO  felt  was  the  most  expert  person  in  any 
particular  subject  aatter  field.  An  example  of  the  interview 
process  is  presented  here  to  show  how  this  was  done. 

Q.  Who  do  you  consider  to  be  an  expert  in  any  specialized  area  in 
our  business? 

A.  The  best  person  on  my  staff  John  Doe  (fictitious)  in  staffing. 
Q.  What  is  it  in  the  staffing  area  that  John  is  expert  at? 

A.  He  really  gets  jobs  filled  in  a  hurry. 


Q.  Do  you  attribute  his  ability  to  get  jobs  filled  fast  to  his 
specialized  knowledge  of  a  particular  aspect  of  staffing?  For 
exaaple,  is  he  unusually  adept  at  reviewing  a  person’s  171  and 
■aking  a  rapid  and  accurate  assessient  of  qualifications? 

A.  No,  but  he  really  knows  how  to  work  the  PPP. 

0.  As  far  as  the  PPP  is  concerned,  do  you  feel  he  knows  lore 
about  it  than  anyone  else  in  Aray?  That  is,  do  you  know  anyone 
else,  whether  they  work  for  you  or  not,  who  knows  the  PPP  inside 
out  and  backward,  so  to  speak,  and  who  John  aight  go  to  for 
ad’  ice  about  the  PPP? 

A.  Yeah,  well  John  soaetiaes  checks  with  Sally  Saith  (fictitious) 
over  at  Zip  Coaaand  when  a  particularly  hard  problea  coaes  up. 
But  on  ay  staff,  John  is  the  best  in  this  area. 

Q.  Do  you  think  that  as  far  as  the  PPP  is  concerned,  you  would 
have  even  aore  expertise  available  if  Nary  Saith  were  on  your 
staff? 

A.  Probably. 

Q.  What  other  areas  do  you  feel  John  .  etc. 

At  this  point,  I  now  have  a  lead  for  a  potential  expert  on  the 
PPP  -  Nary  Saith.  Interviews  will  be  held  with  Nary  Saith’ s  CPO, 
and  with  Nary  Saith  herself.  The  preaise  of  this  technique  is 
that  eventually  we  will  be  able  to  find  those  in  CPO  who  are 
considered  by  others  to  be  expert,  and  who  theaselves  will 
recognize  that  they  have  specialized  knowledges  possessed  by  few 
others  in  DA. 


143  Personnel  Offices  initially  contacted  by  letter. 

131  CPO’s  contacted  by  follow-up  telephone  interviews. 

99  CPO’s  provided  at  least  one  naae  of  a  potential  expert. 


Experts 

Identified 

Subject  aatter 

Identif ie 
than  once 

78 

RAP 

9 

60 

HER  or  LR 

11 

58 

PHAC 

11 

27 

TAD 

4 

26 

TSO 

3 

22 

Other 

0 

Table  2:  Suaaary  of  data  concerning  Aray  Experts. 


Tiae  did  not  perait  ae  to  fully  explore  this  search  technique  for 
all  possible  areas  of  expertise  in  personnel  adain is t rat  ion .  By 
the  tiae  we  got  through  the  first  round  (contact  with  CPO’s),  it 
was  becoaing  evident  that  we  should  build  a  prototype  systea  in 


DA  for  an  expert  system  in  one  specialized  area,  PN&C. 
Therefore,  we  concentrated  our  efforts  on  classifiers.  Results 
of  the  first  tier  search  are  summarized  in  Table  2. 

A  listing  of  the  nominated  experts  is  included  at  Appendix  H  of 
the  original  report  only.  Due  to  the  potential  sensitive  nature 
of  this  information,  I  felt  it  wise  to  include  it  in  only  the 
original  report. 

Some  further  refinement  of  the  identifications  will  be  necessary 
if  Army  decides  to  pursue  the  development  of  a  prototype  expert 
system  in  PN&C.  Since  a  prototype  will  encompass  only  a  block  of 
occupations  in  the  more  than  400  that  exist  in  the  Federal 
Government,  those  experts  who  are  most  knowledgeable  of  these 
occupations  need  to  be  finally  identified,  and  their  availability 
determined.  It  is  not  anticipated  that  the  experts  will  need  to 
be  utilized  for  any  extended  period  of  time.  Much  of  the 
development  work  can  be  accomplished  by  knowledgeable 
classifiers,  and  the  experts  be  asked  to  examine  the  decision 
making  process  possessed  by  the  prototype  in  order  to  refine  and 
improve  upon  its  performance.  Thus  the  experts  need  only  be  used 
in  a  verification  and  refinement  roll.  1  anticipate  this  to  be 
on  a  strictly  voluntary  basis. 
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In  thin  chapter  I  will  explore  the  potential  for  eventual  cost 
savings  by  utilizing  expert  systems  in  civilian  personnel 
administration.  At  this  stage  of  the  process  of  exploring  the 
use  of  expert* systems  it  is  impossible  to  analyze  cost  savings 
with  any  degree  of  accuracy.  There  are  issues  which  drive 
potential  savings,  or  for  that  matter  additional  costs,  which 
have  not  yet  been  decided.  It  is  simply  too  early  in  the  game  to 
expect  these  issues  to  be  decided  which  would  give  a  better  basis 
for  estimating  potential  savings. 

One  of  these  issues  is  whether  or  not  a  system  can  be  fielded 
which  can  be  used  directly  by  supervisors  and  employees  to 
accomplish  personnel  tasks.  I  believe  this  issue  is  more 
correctly  defined  as:  can  a  system  be  fielded  which  will  be 
accepted  by  policy  makers  as  being  sufficient  to  be  used  directly 
by  supervisors  and  employees.  I  am  confident  that  a  system  could 
be  built  that  is  usable  by  other  than  personnel is ts .  The  degree 
to  which  it  will  satisfy  policy  makers  that  it  is  fully 
protecting  ALL  possible  areas  for  manipulation  by  others,  and 
that  it  produces  identical  results  that  under  our  current  system 
may  be  decided  on  the  basis  of  political  or  special  interest 
group  influences  is  much  less  confidently  believed.  In  fact  *1  do 
not  believe  it  is  possible. 

Occasionally  such  issues  as  straight  job  classification  are  the 
result  of  influences  tha't  are  not  totally  related  to  expert 
allocation  of  a  job  classification.  This  is  not  being  cited  as  a 
sour  grapes  observation  since  these  other  influences  are 
sometimes  more  important  than  a  puristic  application  of  rules. 
However,  the  situation  is  a  real  one,  and  I  would  be  the  last  to 
tout  expert  systems  as  being  able  to  accomplish  these  kinds  of 
decision  processes.  They  are  simply  too  fluid  to  be  captured  in 
a  computer  program. 

The  vast  majority  of  daily  activities  at  an  operating  civilian 
personnel  office  do  not  fall  into  these  special  influence 
categories.  However,  there  is  the  issue  of  system  manipulation 
that  could  pose  a  very  real  threat  at  the  operating  level.  If 
someone  sets  out  to  purposefully  cheat  the  system,  for  example  in 
the  classification  of  a  job,  or  the  qualifications  of  a  candidate 
for  a  position,  then  they  will  succeed.  This  is  not  to  say  that 
safeguards  could  not  be  built  into  the  system  to  alert  others  to 
possible  inconsistencies  (cheating)  by  others,  or  to  even  build 
in  safeguards  that  totally  prevent  some  kinds  of  manipulation  to 
be  undertaken.  For  example,  a  PM&C  expert  system  could  be 
designed  that  would  assess  position  management  considerations  of 
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"ideal"  organizational  structures  and  distribution  of  duties 
aaongst  positions  in  the  "most  efficient  manner".  Such  a  design 
would  call  for  the  system  to  reject  any  efforts  to  structure  a 
job  or  the  organization  outside  these  parameters.  Such  an 
attempt  would  cause  the  system  to  notify  the  supervisor  that 
he/she  would  need  to  seek  human  assistance  in  the  personnel 
office.  This,  of  course,  may  lead  to  chaos  if  everyone  is  trying 
to  cheat  the  system  as  some  among  us  believe  supervisors  are. 
Given  this  set  of  circumstances,  an  expert  system  would  be  more 
trouble  than  it  is  worth,  and  would  result  in  no  savings 
whatsoever  (in  fact  additional  costs  in  terms  of  time  and  the 
cost  of  system  development). 

Another  issue  of  importance  in  assessing  costs  is  the  degree  to 
which  policy  makers  can  accept  errors.  We  have  standards  set  for 
humans  in  the  performance  of  personnel  tasks.  For  example,  Army 
expects  a  95*  job  classification  accuracy  rate  by  its  position 
classifiers  (the  task  of  assigning  a  title,  series  and  grade  to 
a  set  of  duties).  This  goal  is  met  with  varying  degrees  of 
success  by  human  classifiers.  Frequently,  being  a  subjective 
outcome,  there  are  differences  of  OPINION  as  to  the  degree  this 
is  being  successfully  accomplished. 

One  could  expect  an  expert  system  to  have  the  same  problem, 
depending  on  who  is  assessing  the  accuracy  of  its  results.  One 
could  even  expect  occasional  errors  by  a  system  based  on 
fallacies  in  its  knowledge  base,  inferencing  techniques  and  the 
like  (all  fixable  in  a  permanent  way  however).  I  do  not  believe 
one  should  expect  an  expert  system  to  he  able  to  provide 
solutions  that  are  acceptable  to  ANY  and  ALL  self  proclaimed 
"experts".  There  will  be  differences  of  opinion  among  human 
experts  in  nearly  all  areas  of  personnel  functions  whether  they 
are  performed  by  human  OR  machine.  I  would  venture  to  say  that 
even  after  years  of  development  and  debugging,  an  expert  system 
will  occasionally  make  "errors"  as  judged  by  humans.  After  all, 
the  system  will  be  designed  to  mimic  these  same  humans,  and  their 
ability  to  make  errors. 

I  have  made  certain  assumptions  in  this  report  about  these  issues 
to  assess  probable  cost  savings.  These  assumptions  are  as 
follows : 

a.  That  supervisors  will  be  entrusted  not  to  try  and  be 
deliberately  dishonest  in  using  an  expert  system  to  accomplish 
personnel  functions. 

b.  That  there  is  some  tolerance  for  error  whether  a 
function  is  performed  by  humans  or  by  machines,  and  that  this 
error  rate  is  acceptable  in  a  machine  as  long  as  it  is 
functioning  at  the  same  level  as  a  human. 

At  Appendix  D  there  is  a  spreadsheet  analysis  of  potential  cost 
savings.  This  analysis  is  based  on  the  number  of  professional 
personnel ists ,  personnel  assistants,  and  clerical  personnel 
assigned  to  personnel  offices  both  operating  and  staff, 


worldwide.  I  obtained  numbers  of  personnel  for  the  past  three 
years  using  reports  that  were  generated  by  CIVPERCINS  for  the 
FORCAST  group,  and  averaged  the  figures.  Figures  for  non-200 
series  personnel  (secretaries,  clerk  typists,  etc.)  are  based  on 
an  estimate  that  used  four  operating  CPO's  current  staffs,  and 
projecting  the  ratio  across  the  board  to  all  personnel  offices. 
Current  costs  are  based  on  step  4  of  the  salary  scale  for  the 
grades  and  number  of  personnel  using  the  1985  pay  table.  There 
are  133  operating  CPO’s  in  CONUS,  43  operating  offices  overseas, 
and  40  staff  offices  for  a  total  of  216  offices  worldwide. 

Perhaps  the  most  controversial  aspect  of  this  analysis  is  the 
percent  replace  estimate.  It  is  here  that  I  could  not  build  an 
objective  case  for  the  number  I  used.  My  general  approach  at 
selecting  this  number  is  as  follows: 

There  are  certain  functions  that  COULD  be  more  amenable  than 
others  to  expert  systems  applications  in  which  the  system 
performs  the  function  of  the  human.  I  estimate  that  an  expert 
system  could  replace  all  operating  office  position  classifiers 
and  staffing  specialists  (please  keep  in  mind  my  assumptions 
above).  Thus,  in  a  full-blown  expert  system,  supervisors  would 
use  it  at  the  operating  level  to  obtain  job  classification, 
position  management  and  other  functions  provided  by  position 
classifiers.  They  would  also  use  it  to  fill  their  jobs  which 
would  evaluate  the  qualifications  of  candidates,  make  referrals 
of  best  qualified,  and  take  peripheral  actions  such  as  setting 
pay,  making  sure  priority  placement  considerations  are  taken, 
etc.  To  expound  on  the  details  of  each  of  these  tasks,  and  how 
it  would  be  done  is  outside  the  scope  of  this  project.  At  this 
stage  of  research,  given  the  fact  that  experimentation  and 
development  of  such  a  system  will  take  several  years  to  prove  or 
disprove  any  of  the  possibilities  that  exist,  it  would  be 
foolhardy  of  me  to  try  and  prove  these  ideas.  However,  given  the 
fact  that  expert  systems  can  be  developed  (and  have  been)  that 
perform  human  judgmental  tasks  of  an  equal  or  more  difficult 
level,  I  believe  it  is  reasonable  to  postulate  that  the  potential 
for  totally  replacing  operating  office  classifiers  and  staffing 
specialists  is  worth  considering,  and  is  potentially  possible. 

As  can  be  seen,  some  of  the  functional  specializations  do  not 
have  as  dramatic  a  potential  savings  figure  attached.  For 
exasole  in  the  area  of  labor  relations,  there  are  activities 
perforiiied  by  these  specialists  such  as  negotiations  with  labor 
unions  that  cannot  be  supplanted  with  a  computer  which  would  do 
the  same  thing.  As  mentioned  in  a  previous  chapter,  expert 
systems  could  greatly  assist  the  labor  relations  specialist  in 
preparing  to  negotiate,  and  assist  this  human  in  better  doing  the 
job,  but  I  have  not  attempted  to  assess  the  cost  benefits  of  this 
kind  of  uses  of  expert  systems.  At  this  stage  of  the  research 
process,  speculation  in  to  areas  of  this  kind  of  expert  systems 
usage  is  much  less  defensible  than  the  notion  of  actually 
replacing  people  with  a  system  that  does  traditional  people  work. 

Total  costs  savings  COULD  amount  to  nearly  $109  million  per  year 


based  on  these  assumptions 


I  realize  that  there  are  those  who  are  extremely  uncomfortable 
with  this  kind  of  analysis.  I  would  be  too  if  the  potential 
savings  had  to  be  totally  proven  and  unquestionably  justify 
further  developmental  efforts.  Because  of  the  very  minimal 
amount  of  investment  required  to  be  able  to  further  explore  these 
necessarily  vague  potentials  (which  are  tremendous),  I  believe 
that  as  a  minimum,  the  next  phase  of  R&D  work  is  justified.  A 
prototype  system  in  the  area  of  position  classification  will  cost 
in  the  neighborhood  of  $1.5M  to  $3.0M  (depending  on  what  method 
is  chosen  to  develop  it).  With  a  design  goal  of  having  it 
actually  replace  human  classifiers,  and  an  acceptance  on  the  part 
of  the  policy  makers  of  the  assumptions  above,  hard  evidence  can 
then  be  gathered  to  support  the  somewhat  vague  potentials 
expressed  herein.  To  do  less,  simply  is  not  justified  by  the 
claim  that  total  proof  is  required  before  R&D  activities  are 
initiated.  We  would  never  have  explored  space,  never  have 
eradicated  the  world  of  polio  or  a  host  of  other  things  if  R&D 
efforts  had  not  been  undertaken  without  provable  benefits  at  the 
outset . 
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There  is  a  very  real  potential  for  considerable  impact  on 
currently  employed  and  future  personnelists  if  expert  systems  are 
utilized  in  the  civilian  personnel  administration  functions.  If 
we  are  able  to  replace  traditional  positions  performed  by  humans 
with  computer  systems  that  perform  the  same  functions,  then  there 
exists  a  need  to  address  these  issues  early  in  the  planning 
stages  from  several  points  of  view. 

Some  would  argue  that  we  should  not  employ  the  use  of  expert 
systems  not  on  the  basis  of  questioning  their  ability  to  do  the 
rules  applications  functions  that  personnelists  perform,  but  on 
the  basis  that  some  jobs  are  a  unique  combination  of  rules 
appliers,  and  catalysts  for  getting  things  done  that  do  not  rely 
solely  on  applying  rules.  There  are  human  interactions  with 
people  that  are  an  essential  ingredient  to  everything 
personnelists  do,  that  cannot  be  performed  by  any  machine,  no 
matter  how  smart  it  is.  The  human  in  the  job  makes  the  job  what 
it  is,  not  simply  a  system  that  knows  all,  applies  all  rules 
equally,  and  is  painfully  exact  and  consistent. 

We  have  all  been  exposed  to  situations  in  which  the  ability  of 
the  person  in  the  job  to  create  a  unique  solution  to  a  problem  or 
the  ability  to  apply  the  human  touch  to  a  particular  situation 
was  the  only  reason  it  succeeded.  For  example,  I  know  a  case 
(really  not  unique)  in  which  a  management  employee  relations 
specialist,  with  his  ear  to  the  ground,  was  aware  of  growing 
morale  problems  in  an  organization.  They  were  not  obvious 
problems  such  as  grievances  and  disciplinary  situations,  but 
"rumblings"  that  only  an  astute,  dedicated  MER  person  would  be 
aware.  He  was  aware  of  the  probable  causes  of  the  problem  (a 
supervisor  who  was  practicing  favoritism  with  some  employees), 
and  watched  the  situation  unfolding.  There  was  never  a  reason  to 
approach  this  supervisor  with  "assistance"  since  it  was  not 
serious  enough  to  trigger  such  a  response  for  the  MER. 

It  so  happened  that  one  day  a  SF-52  floated  into  the  office  that 
would  create  a  new  kind  of  job  for  this  supervisor’s 
organization,  and  the  MER  was  astute  enough  and  knowledgeable 
enough  of  the  environment  that  he  was  instantly  aware  that  the 
supervisor  was  creating  an  advancement  job  for  a  favored 
employee.  He  was  also  aware  that  this  would  probably  be  the 
straw  that  would  break  the  camel’s  back,  and  serious  problems 
would,  in  all  likelihood,  erupt  in  the  organization.  It  was  also 
fairly  obvious,  that  the  newly  created  job  would  not  raise  flags 
to  the  PM&C  people  since  it  was  consistent  with  mission  and 
organization  of  the  supervisor’s  office. 


With  knowledge  that  only  a  human  could  practically  possess,  the 
MER  began  to  diffuse  the  problem  before  it  could  erupt  by 
providing  "positive  management  advisory  services".  In  other 
words,  the  MER  developed  a  strategy  to  prevent  the  situation  from 
blowing  up.  I  won’t  go  into  details  of  how,  since  it  is 
irrelevant  to  the  point  I  want  to  make.  The  fact  is,  this  MER 
took  steps  to  prevent  a  problem  before  it  occurred.  Some  would 
argue  that  this  is  not  a  function  of  CPO.  Others  would  argue 
that  this  was  meddling  on  the  part  of  the  MER,  and  there  is  no 
proof  that  the  undesirable  outcome  would  have  occurred  had  the 
MER  left  things  alone. 

Irrespective  of  which  side  one  takes,  it  is  clearly  evident  that 
an  expert  system  would  not  have  been  able  to  accomplish  what  the 
human  MER  accomplished.  Therefore,  one  could  not  expect  to 
replace  people  with  systems  in  which  the  human  involvement  aspect 
is  critical.  In  this  respect,  one  could  argue  that  everything 
ANY  personnelist  is  expected  to  do  must  be  accomplished  with  an 
eye  toward  human  involvement  if  it  is  justified.  If  this  is  so, 
then  expert  systems  have  no  place  in  replacing  personnel ists  in 
the  personnel  office.  If  it  is  so  only  on  an  ideal,  theoretical 
basis  (i.e.  one  would  hope  that  all  personnel ists  are  at  all 
times  performing  their  jobs  with  appropriate  human  involvement), 
and  in  reality  it  is  so  only  some  of  the  time,  then  expert 
systems  deserve  the  opportunity  to  compete  with  personnel ists  for 
their  jobs.  At  least  those  jobs  in  which  it  is  not  routinely 
expected  that  personnel ists  will  so  function  to  a  high  degree  of 
efficiency. 

But  what  of  these  people  who  have  been  highly  trained,  integrated 
into  the  personnel  administration  functions  so  tightly,  and  who 
become  excess  due  to  the  ability  of  the  machine  to  replace  them? 
Should  there  be  an  uprising  of  defensiveness  that  undermines  the 
potential  uses  of  expert  systems  so  that  they  aren’t  given  a 
chance  of  proving  or  disproving  their  worth?  Should  the  threat 
of  job  loss  cause  those  few  who  are  marginally  performing  their 
"human  only"  jobs  to  quickly  become  "human"  and  perform  those 
ideal  functions  we  expect  of  them?  Should  the  identifiable  areas 
in  which  it  is  absolutely  essential  that  humans  participate 
control  the  entire  nature  of  our  functions  such  that  there  are 
"no  areas  in  which  the  human  is  replaceable"? 

Assuming  that  there  are  areas  in  which  functions  can  be  carved 
out  to  be  performed  by  machines,  there  is  the  very  real  problem 
of  what  to  do  with  excess  personnel.  It  could  be  that  as  pieces 
of  our  business  are  automated,  jobs  are  changed  in  such  a  way 
that  they  do  not  resemble  our  traditional  jobs.  Straight  job 
classification  may  cease  to  exist  for  humans,  yet  there  are 
pieces  of  the  PM&C  jobs  that  might  not  lend  themselves  to 
automation.  People’s  thinking  about  what  a  PM&C  person  does 
would  need  to  be  radically  adjusted.  The  PM&C  person’s  view  of 
his/her  job  would  need  to  be  likewise  amended.  For  example,  the 
machine  classifies  the  job,  the  PM&C  person  is  not  equipped  to 
second  guess  the  classification,  but  must  use  the  result  to 
assist  the  supervisor  in  other  areas  such  as  position  management. 


70 


If  in  fact  the  system  can  perform  position  management  functions 
also,  the  PM&C  person’s  roll  changes  even  more  dramatically. 

These  issues  will  challenge  our  leaders  in  the  future  if  expert 
systems  begin  to  become  integrated  into  the  personnel  functions 
and  take  over  tasks  performed  by  personnelists .  Challenges  to 
restructure  the  way  we  conduct  our  business  will  be  greater  than 
has  ever  faced  us  in  the  past.  I  do  not  believe  it  forces  us 
into  a  protectionism  mode.  We  need  neither  protect  the  number  of 
traditional  personnelist  jobs  that  exist  today,  nor  protect  the 
tasks  that  lend  themselves  to  expert  systems  technology.  People 
have  the  unique  capability  of  adjusting  to  changing  environments 
and  situations.  This  technology  in  itself  offers  alternatives 
to  our  current  jobs.  All  advances  that  have  been  made  by  mankind 
have  somehow  created  opportunities  for  those  who  have  the 
willingness  to  accept  the  challenges. 


As  mentioned  previously,  Mr.  Severin  Johnson,  CPO,  Oakland  Army 
Base,  was  detailed  to  work  with  me  on  this  project  for  a  period 
of  time.  He  accomplished  most  of  the  work  associated  with 
locating  Army  experts.  He  also  conducted  the  development  of  an 
expert  system  using  Expert  Ease  on  an  experimental  basis.  This 
served  two  purposes.  First,  it  helped  to  give  insight  into  how 
quickly  a  professional  personnelist  with  very  little  background 
in  using  computers  could  assimilate  enough  skill  and  ability  to 
use  the  computer  to  do  expert  system  development  work.  Second, 
it  provided  an  opportunity  to  develop  another  application  of  this 
technology  in  the  field  of  job  classification,  and  dealt  more 
extensively  with  GS  jobs  in  the  process.  I  was  definitely 
encouraged  by  the  results  of  this  experiment. 

Mr.  Johnson  chronicles  his  work  in  Appendix  G.  It  is  presented, 
unedited,  therein. 


73 


REFERENCES 


Anderson,  John  R. ,  Brian  J.  Reiser.  "The  LISP  Tutor."  BYTE 
Magazine,  Vol.  10,  No.  4,  April  1985. 

Bachant,  Judith,  John  McDermott.  "R1  Revisited:  Four  Years  in  the 
Trenches."  AI  Magazine,  Vol.  5,  No.  3,  Fall,  1984. 

Barr,  Avron,  Edward  A.  Feigenbaum,  Eds.  The  Handbook  of 
Artificial  Intelligence,  Volumes  I  -  III.  William  Kaufmann, 
Inc.,  Los  Altos,  CA,  1981. 

Barstow,  David.  "Artificial  Intelligence  at  Schlumberger . ”  AI 
Magazine,  Vol.  5,  No.  4,  Winter  1985. 

Bierre,  Pierre.  "The  Professor’s  Challenge."  AI  Magazine,  Vol.  5, 
No.  4,  Winter  1985. 

Biggs,  Rick.  "Knowledge  Representation  in  Sanskrit  and  Artificial 
Intelligence."  AI  Magazine,  Vol.  6,  No.  1,  Spring  1985. 

D'Ambrosio,  Bruce.  "Expert  Systems-Mvth  or  Reality."  BYTE 
Magazine,  Vol.  10,  No.  1,  January,  1985. 

D'Ambrosio,  Bruce.  "Building  Expert  Systems  with  M.l."  BYTE 
Magazine,  Vol.  10,  No.  6,  June  1985. 

Darany,  Theodore  S.  "Computer  Applications  to  Personnel 
(Releasing  the  Genie  -  Harnessing  the  Dragon)."  Public  Personnel 
Management  Journal,  Vol.  13,  No.  4,  Winter  1984. 

Davis,  Dwight  B.  "English:  The  Newest  Computer  Language."  High 
Technology,  February,  1984. 

Deering,  Michael  F.  "Architectures  for  AI."  BYTE  Magazine,  Vol. 
10,  No.  4,  April  1985. 

Doherty,  Michael  J.  "Comments  on  Sixth  Generation  Computers."  Dr. 
Dobb’s  Journal,  #92  June,  1984. 

Doyle,  Jon.  "Expert  Systems  Without  Computers  Or  Theory  and  Trust 
in  Artificial  Intelligence."  AI  Magazine,  Vol.  5,  No.  2,  Summer 
1984. 

Faggin,  Frederico.  "The  Challenge  of  Bringing  New  Ideas  to 
Market."  High  Technology,  February,  1985. 

Feldman,  Jerome  A.  "Connections."  BYTE  Magazine,  Vol.  10,  No.  4, 
April  1985. 

Grigonis,  Richard.  "Sixth  Generation  Computers."  Dr.  Dobb’s 
Journal,  #91  May,  1984. 


A-l 


Grigonis,  Richard.  "And  Still  More  Fifth  Generation  Computers!" 
Dr.  Dobb’s  Journal,  #82  August,  1983. 

Harmon,  Paul,  David  King.  Expert  Systems.  John  Wiley  &  Sons,  New 
York.  1985. 

Harris,  Larry  R.  "Experience  with  INTELLECT:  Artificial 
Intelligence  Technology  Transfer."  AI  Magazine,  Vol.  5,  No.  2, 
Summer  1984. 

Hart,  Peter  E.  "Artificial  Intelligence  in  Transition."  AI 
Magazine,  Vol.  5,  No.  3,  Fall,  1984. 

Hayes-Roth,  Frederick,  Donald  A.  Waterman,  Douglas  B.  Lenat. 
Building  Expert  Systems.  Addison-Wes ley ,  Reading,  Mass.,  1983. 

Hewitt,  Carl.  "The  Challenge  of  Open  Systems."  BYTE  Magazine, 
Vol.  10,  No.  4,  April  1985. 

Hinton,  Geoffrey  E.  "Learning  in  Parallel  Networks."  BYTE 

Magazine,  Vol.  10,  No.  4,  April  1985. 

Hollan,  James  D.,  Edwin  L.  Hutchins,  Louis  Weitzman.  "STEAMER:  An 
Interactive  Inspectable  Simulation-Based  Training  System."  AI 
Magazine,  Vol.  5,  No.  2,  Summer,  1984. 

Kaplan,  S.  Jerrold.  "The  Industrialization  of  Artificial 
Intelligence:  From  By-Line  to  Bottom  Line."  AI  Magazine,  Vol.  5, 

No.  2,  Summer  1984. 

Katke,  William.  "Learning  Language  Using  a  Pattern  Recognition 
Approach."  AI  Magazine,  Vol.  6,  No.  1,  Spring  1985. 

Kinnucan,  Paul.  "Computers  that  Think  Like  Experts."  High 
Technology,  January,  1984. 

Kunz,  John  C.,  Thomas  P.  Kehler,  Michael  D.  Williams. 
"Applications  Development  Using  a  Hybrid  AI  Development  System." 
AI  Magazine,  Vol.  5,  No.  3,  Fall  1984. 

Laurent,  Jean-Pierre.  "AI  Research  in  France."  AI  Magazine,  Vol. 
6,  No.  1,  Spring  1985. 

Lerner,  Eric  J.  "Why  Can’t  a  Computer  be  More  Like  a  Brain?"  High 
Technology,  August,  1984. 

Martindale,  Loren  D.  "The  Case  for  Expert  Systems  in  Personnel 
Administration."  The  Nick  Hoag  Award  for  Professional 
Development,  Department  of  Army  1985. 

McCalla,  Gordon,  Nick  Cercone.  "Artificial  Intelligence  in 
Canada:  A  Review."  AI  Magazine,  Vol.  5,  No.  4,  Winter  1985. 

Meltzer,  Bernard.  "Knowledge  and  Experience  in  Artificial 
Intelligence."  AI  Magazine,  Vol.  6,  No.  1,  Spring  1985. 


A- 2 


Michaelsen,  Robert  H. ,  Donald  R.  Michie,  Albert  Boulanger.  "The 
Technology  of  Expert  Systems."  BYTE  Magazine,  Vol.  10,  No.  4, 
April  1985. 

Mostow,  Jack.  "Toward  Better  Models  of  the  Design  Process."  A1 
Magazine,  Vol.  6,  No.  1,  April  1985. 

Nilsson,  Nils  J.  "Artificial  Intelligence,  Employment  and 
Income."  AI  Magazine,  Vol.  5,  No.  2,  Summer  1984. 

Niwa,  Kiyoshi,  Koji  Sasaki,  Hirokazu  Ihara.  "An  Experimental 
Comparison  of  Knowledge  Representation  Schemes."  AI  Magazine, 
Vol.  5,  No.  2,  Summer  1984. 

Park,  Jack.  "Expert  Systems  and  the  Weather."  Dr.  Dobb ’ s  .Journal , 
♦90  April,  1984. 

Pecora,  Vincent  J.  ,  Jr.  "EXPRS  A  Prototype  Expert  System  Using 
Prolog  for  Data  Fusion."  AI  Magazine,  Vol.  5,  No.  2,  Summer  1984. 

Polit,  Stephen.  "R1  and  Beyond:  AI  Technology  Transfer  at  DEC." 
AI  Magazine,  Vol.  5,  No.  4,  Winter  1985. 

Raike,  William  M.  "The  Fifth  Generation  in  Japan."  BYTE  Magazine, 
Vol.  10,  No.  4,  April  1985. 

Rauch,  Herbert  E.  "Probability  Concepts  for  an  Expert  System  Used 
for  Data  Fusion."  AI  Magazine,  Vol.  5,  No.  3,  Fall  1984. 

Schank,  Roger,  Larry  Hunter.  "The  Quest  to  Understand  Thinking.” 
BYTE  Magazine,  Vol.  10,  No.  4,  April  1985. 

Smith,  Reid  G.  "On  the  Development  of  Commercial  Expert  Systems." 
AI  Magazine,  Vol.  5,  No.  3,  Fall  1984. 

Stacy,  E.  Webb,  Ruth  Ann  Marco,  James  Hassett,  Jean  MacMillan, 
Craig  Cook,  Bruce  Isreal,  Darrell  Hubbard.  "The  Uses  of 
Artificial  Intelligence  in  Military  Personnel  Offices."  Report 
TR-1093-1  Prepared  for  U.S.  Army  Ballistic  Research  Laboratory  by 
Scientific  Systems,  1  April  1985. 

Stevens,  John  K.  "Reverse  Engineering  the  Brain."  BYTE  Magazine, 
Vol.  10,  No.  4,  April  1985. 

Thompson,  Beverly  A.,  William  A.  Thompson.  "Inside  an  Expert 
System."  BYTE  Magazine,  Vol.  10,  No.  4,  April  1985. 

Touretzky,  David  S.  LISP  A  Gentle  Introduction  to  Symbolic 
Computation.  Harper  &  Row,  New  York,  1984. 

Weiss,  Sholom  M. ,  Casimir  A.  Kulikowski.  A  Practical  Guide  to 
Designing  Expert  Systems.  Rowman  &  Allanheld,  Totowa,  N.J.,  1984. 


Wasseraan,  Kenneth.  "Physical  Object  Representation  and 
Generalization:  A  Survey  of  Programs  for  Semantic-Based  Natural 

Language  Processing."  AI  Magazine,  Vol.  5,  No.  4,  Winter  1985. 

Wilensky,  Robert.  LISPcraft.  W.W.  Norton,  New  York,  1984. 

Winston,  Patrick  H.  "The  LISP  Revolution."  BYTE  Magazine,  Vol. 
10,  No.  4,  April  1985. 


APPENDIX  ■ 


EQUIPMENT  USED  ON  THIS  PROJECT 


This  appendix  lists. and  describes  the  software  and  hardware  used 
on  this  project  for  those  who  may  be  interested.  This  equipment 
cost  a  total  of  $9,012.44.  It  was  invaluable  to  me  in  two  main 
ways.  First  it  gave  me  the  means  to  efficiently  maintain  data 
and  information  collected,  and  to  manipulate  the  data  in  an 
analytical  way.  Since  I  performed  my  own  administrative  support 
to  include  typing,  collecting  and  storing  information  and  data, 
and  other  clerical  tasks,  I  would  not  have  been  able  to  do  as 
much  without  either  a  clerical  support  person,  or  this  equipment. 
Second  it  gave  me  a  tool  to  directly  evaluate  expert  systems 
technology  that  could  not  have  been  done  without  the  equipment. 
All  of  this  equipment,  with  the  possible  exception  of  the  Expert 
Ease  software,  can  be  further  utilized  within  the  Department  of 
Army  in  nearly  any  office  or  shop  setting.  This  system  will 
continue  to  be  a  useful  and  productive  tool  to  some  office  for 
years  to  come. 

Product  Use 

Compaq  Plus  Microcomputer  System  hardware  to  run  all  software 

products.  Also  used  for  presenting 
briefings.  Since  it  is  a  transpor¬ 
table  unit,  it  can  be  taken  on  trips 

.  Epson  LQ-1500  Printer  Printing  correspondence,  reports, 

listings,  mailing  labels,  programs 
and  this  report  in  it's  entirety. 

Princeton  Color  Monitor  Development  and  presentation  of 

briefings,  and  to  provide  a  stand¬ 
ard  size  monitor  for  other  applic¬ 
ations  . 

Wordstar  Pro  Software  for  word  processing, 

spelling  check  and  indexing.  (The 
workhorse  I  used  most  often.) 

KnowledgeMan  Data  Base  Manager  for  several 

data  bases. 

SuperCalc3  Spreadsheet  for  analysis  of 

numerical  and  costs  data. 

Sidekick  On-line  utility  program  for  notes 

calculations  in  Hex,  ASCII  tables 
and  other  programming  aids. 

BASICA  Programming  language  for  briefing 

program,  other  utilities  when 
needed . 
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Expert  Ease 


Expert  system  development'  tool  for 
assessment  of  expert  systems  tech¬ 
nology,  and  development  of  demon¬ 
stration  module. 


Here  is  a  thumbnail  assessment  of  this  equipment  for  any  who  are 
interested.  If  further  details  are  desired,  please  contact  me. 
The  Compaq  is  a  very  good  IBM  PC  compatible.  It  gave  me  no 
problems  throughout  the  project,  and  was  run  for  8  to  10  hours 
per  day.  All  of  the  software  ran  flawlessly  on  this  machine  with 
the  exception  of  Expert  Ease,  which  could  not  be  copied  to  the 
hard  disk.  The  printer  worked  excellently,  but  used  ribbons  up 
pretty  fast.  As  you  can  see,  this  dot  matrix  printer  puts  out 
near  letter  quality  print.  In  draft  mode,  it  really  zips  along, 
and  in  LQ  mode  it  isn’t  a  slouch  either,  actually  faster  than 
most,  formed  character  printers  in  comparable  price  range.  The 
Princeton  monitor  gave  good  graphics,  and  no  trouble.  Wordstar 
is  an  excellent  wordprocessor .  I  have  used  Wordstar  for  several 
years.  A  bit  hard  to  learn,  but  in  terms  of  functions  offered, 
it  has  no  equals.  Its  spelling  checker  is  fast  and  easy  to  use. 
The  Cadillac  of  microcomputer  wordprocessors .  KnowledgeMan  is 
difficult  to  learn  to  use,  primarily  because  of  its  manual. 
However  it  is  a  very  powerful  DBMS  (much  more  so  than  dBase  II  or 
III,  R:Base,  etc.).  I  wouldn’t  recommend  it  to  anyone  unless  you 
are  willing  to  struggle  through  the  learning  curve.  However,  if 
you  can  get  your  SCPMIS  database  downloaded  to  IBM  PC  compatible 
disks,  then  KMan  is  the  only  DBMS  that  can  practically  handle 
such  a  file.  SuperCalc3  is  an  old  favorite  qf  mine.  Probably 
the  best  spreadsheet  on  the  market  (although  I  haven’t  used  Lotus 
1,2,3).  SC3  has  justifiably  received  good  ratings  from  the 
media,  and  gives  good  graphs  in  the  blink  of  an  eye.  Expert  Ease 
was  assessed  in  the  report,  so  I  don’t  have  anything  more  to  say 
about  it  except  that  there  have  been  new  microcomputer  expert 
system  development  tools  introduced  since  this  product  was  bought 
which  are  better  and  cheaper. 
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TINE  SCHBOULB  FOB  PHOTOTYPE  DEVELOPMENT 


Date 

Event 

Comments 

Start 

Project  approval 

Approval  of  project 
by  DA,  OPM,  etc. 
Includes  funds  and 
manpower  spaces  app¬ 
roval,  project  site 
approval,  reporting 
chain  established. 

Start+30 

Project  Officer  Selected 

DA  selection  made 

Start+60 

Project  Officer  report  to 
project  site  installation. 

Start+90 

Office  set  up,  staffing  of 
project  office  initiated. 

Project  officer  to 
secure . 

Start+150 

Project  office  in  opera¬ 
tion,  programmer  position 
filled,  ADP  approval  sec¬ 
ured. 

Start+170 

Procurement  requisitions 
prepared  fort  hardware, 
and  software. 

Ready  for  procure¬ 
ment  action. 

Start+180 

Project  officer  and  prog¬ 
rammer  begin  crash  course 
to  learn  LISP  and  use  of 
expert  system  tool. 

At  appropriate  sites 

Start+180 

Begin  work  on  PM&C  sys¬ 
tem  at  project  site. 

Start+200 

ADP  hardware  and  software 
rece i ved . 

Start+320 

Selections  made  for  Pers¬ 
onnel  ists-programmers , 

Elec  Tech,  Secretary. 

Start+350 

Personnel ists-programmers , 
Elec  Tech,  Admin  Asst,  and 
Secretary  on  board. 

Start+350 

Other  team  members  begin 
training  in  use  of  equip. 

At  project  office 
and  training  sites 

Start+450 

Start+480 

Start+540 

Start+720 

Start+720 

Start+720 

Start+1080 

Start+1080 


Phase  I  of  the  PM&C 
prototype  ready  for 
val idat ion . 


Phase  I  validations 
complete . 

Phase  II  of  PM&C 
prototype  ready  for 
validation . 

All  validations 
complete. 

Begin  user  test  of 
system  in  using  organ¬ 
ization. 

Begin  evaluation  of 
system  in  use. 

User  test  complete. 

Begin  final  evaluation 
of  prototype 


Includes  at  least 
three  GS  type  jobs, 
for  which  any  grade 
can  be  accurately 
assigned  by  system. 

Other  Army  experts 
used . 

Includes  remainder 
of  jobs  in  test 
organization . 


System  capable  of 
classifying  any  job 
in  test  organization 


Start+1170 


Final  report  on  prototype 
test  provided  to  Army.  » 


es- 

Nuaber 


CPO’s  and  PHS’s 
Currently 
Percent  replace 
Needed  tt/Ex  Sys 


Savings 

CLASSIFIERS 
Currently 
Percent  replace 
Needed  w/Ex  Sys 


LABOR  REL  SPECs 


Percent  replace 
Needed  sa/Ex  Sys 


4787970  120 

90 

478797  12 

4309173  108 


350  10156300  70 

901  90 


STAFFING  SPECS 

Currently 

420 

12187560 

130 

Percent  replace 

90J 

90 

I 

Needed  u/Ex  Sys 

42 

1218756 

13 

311805 

Savings 

378 

10963304 

117 

2306245 

MGT  EHP  REL  SPEC 

Currently 

135 

3917430 

25 

599625 

Percent  replace 

90Z 

90 

: 

Needed  u/Ex  Sys 

13.5 

391743 

2.5 

59962.5 

Savings 

121.5 

3525637 

22.5 

539662.5 

EMPL  DEV  SPECS 

Currently 

95 

2756710 

30 

719550 

Percent  replace 

9UI 

90 

X 

Needed  u/Ex  Sys 

9.5 

275671 

3 

71955 

m 


PENS  CL*  i  ASST 
Currently 
Percent  replace 
Needed  a/ Ex  Sys 
Savings 

OTHER  CLERICAL 
Currently 
Percent  replace 
Needed  w/E<  Sys 


676732.5 


Totals 

Nutber  Cost 


Totals 


Nutber  Ci 

)St 

PERS  CLK  &  ASST 

Nuiber 

Cost 

CPO’s  and  PHS’s 

Currently 

2270 

352S5105 

Currently 

1062 

$35,658,116 

Percent  replace 

Percent  replace 

Needed  w/Ex  Sys 

575.75 

9011653. 

Needed  w/Ex  Sys 

587.8 

$23,456,192 

Savings 

1694.25 

26243453 

Savings 

474.2 

$12,201,924 

OTHER  CLERICAL 

CLASSIFIERS 

Currently 

312 

4351323 

Currently 

819 

$25,918,437 

Percent  replace 

Percent  replace 

Needed  w/Ex  Sys 

176.25 

2496472. 

Needed  sa/Ex  Sys 

98.5 

$3,322,297 

Savings 

135.75 

1855357. 

Savings 

720.5 

$22,596,141 

STAFFING  SPECS 
Currently 

967 

$29,581,440 

Percent  replace 
Needed  w/Ex  Sys 

.  110.2 

$3,530,054 

Savings 

356.3 

$26,051,337 

OVERALL  SUMMARY 

HGT  EHP  REL  SPEC 
Currently 

320 

$10,255,168 

CURRENTLY 

Percent  replace 

Total  Professionals  = 

3647  $117,257,756 

Needed  w/Ex  Sys 

38.5 

$1,335,366 

Total  all  staff  : 

6229 

$156,864,689 

Savings 

281.5 

$8,919,302 

ESTIMATED  NEEDS 

ENPL  DEV  SPECS 
Currently 

309 

$10,037,669 

Under  Expert  Syste* 

1723.25 

$43,033,153 

Percent  replace 

POTENTIAL  SAVINGS 

Needed  w/Ex  Sys 

36.75 

$1,278,905 

Under  Expert  Syste* 

4505.75  $108,331,531 

Savings 

272.25 

$8,758,765 

LABOR  REL  SPECS 
Currently 

170 

$5,306,926 

Percent  replace 
Needed  w/Ex  Sys 

99.5 

$3,602,222 

Savings 

70.5 

$2,204,705 
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DEMONSTRATION  MODULE  FILES 


Printouts  of  files  used  in  the  Expert  System  Demonstration  Module 
are  included  in  this  appendix.  These  files  are  printed  out  in 
Expert  Ease  format.  Three  printouts  for  each  file  are  included. 
They  are:  Attribute  listing,  Examples  listing  and  Rule  listing 
for  each  Problem.  See  appropriate  section  in  the  report  which 
describes  how  these  files  are  used  by  the  demo  module.  NOTE:  Due 
to  the  large  number  of  examples  (188)  used  for  the  Air 
Conditioning  Equipment  Mechanic,  its  Examples  listing  is  not 
included  in  this  appendix. 


EXPERT-EASE  Attribute  Listing,  Proble»:  WA6E.SRA0  Date:  25-feb-85 


faaiiy  :  SAhich  of  the  following  best  describes  the  aajor  tasks  assigned  to  tnis 
position? 

jaint.rpr  :  Maintenance  or  repair  of  equipaent. 

food. prep  :  Preparation  or  serving  of  rood, 

equip. opr  :  Operation  of  equipment. 

cat.rpr  :  What  type  of  equipment  is  aaintained  or  repaired? 
autoaotiv  :  Autoaotive  (to  include  heavy  aobile  equipaent). 

aircono  :  Air  Conditioning  Equipaent. 

cat.pr  :  what  type  of  equipaent  is  operated? 

aot.veh'  :  Motor  vehicles  (i.e.  autoaobiles  and  trucks) 

fork. lft  :  Forklifts  of  any  site: 

crane  :  Cranes  of  any  site. 

class  : 

Ifood.prep  : 
fautoa.rpr  : 
tao.ven.op  : 
ffk.lft.op  : 
icrane.op  : 
tair.c.aec  : 


!XP£RT-EAS£  Exaiple  Listing,  Probie*:  HA6E.5RAD  Date:  25-feb-85 


raniy 

logical 


1  laint.ror 

2  laint.rpr 

3  food. prep 

4  equip. opr 

5  eouip.opr 
o  equip. opr 


cat. rpr  cat. opr 

logical  logical 


autoaotiv  * 

aircono  » 

4  t 

*  aot.veft 

*  fork.ift 

t  crane 


ciass 

logical 


lautoi.rpr 

lair.c.iec 

Ifood.prep 

lao.ven.op 

ifk.lft.op 

Icrane.oo 


EXPERT-EASE  Ruie  Listing,  Probiei:  WAGE. GRAD 


Date:  25-fe6-35 


faaiiy 

lamt.rpr  :  cat. ror 

autciotiv  :  lautoa.rpr 
aircond  :  lair.c.iec 
food. prep  :  Ifooo.prep 
equip. opr  :  cat. opr 

aot.ven  :  iao.ven.op 
'orx.ift  :  ffk.lft.oo 
crane  :  icrane.op 
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EXPERT-EASE  Attribute  Listing,  Prooiei:  AUTOH.RPR  Date:  25-feb-85 


v' 

I 


sq.xino  :  What  king  of  equipment  will  tnis  eepioyee  worx  with? 

nv.nob.eq  :  Heavy  aobile  equipeent  such  as  bulldozers,  roao  graders,  crawier 

tractors,  power  snoveis,  iocoiotives.  coabac  tanks,  cranes,  large 
nssiie  transporters,  and  fire  trucks. 

autos  :  Autoiotive  equipeent  such  as  passenger  cars,  picx-up  trucxs,  ouses, 

sen-trailer  truck  tractors,  warehouse  tractorls,  fan  tractors,  fork¬ 
lifts- motorcycles,  ano  light  coibat  vehicles  sucn  as  jeeps  and 
trucks. 

leveil  ;  Does  the  eepioyee  have  to  have  the  knowledge  to  at  least  check  and  replace 
sparx  plugs,  tail  pipes,  hoses  and  fan  belts? 
seal. rpr  Yes. 


,  no.ror  :  Ho. 

;evel2  :  Does  tn is  eepioyee  have  to  have  the  knowledge  to  at  least  take  tune-uos 
on  venicles? 
tun. ups  :  les. 

no. tunup  :  ho. 

ciass  :  Sorry,  tne  syste*  is  not  yet  capable  of  classifying 
liob.eq.sv  : 
tautc.aecn  : 

nv.j.eq.i  :  Heavy  Mobile  Equipaent  Mechanics. 
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EXPERT-EASE  ExaiPie  Listing,  Problem  AUTOH.8PR  Date:  25-feb-3! 


an. Kina 

ieveil 

levei2 

class 

Logical 

logical 

'logical 

logical 

l 

bv.sob.eq 

no.rpr 

r 

iiob.eq.sv 

riv.sob.eq 

siai.rpr 

no.tunup 

iiob.eq.sv 

3 

hv.iob.eq 

siai.rpr 

tun. ups 

hv.a.eq.i 

4 

autos 

no.rpr 

t 

iiob.eq.sv 

5 

autos 

siai.rpr 

no.tunup 

iiob.eq.sv 

b 

autos 

siai.rpr 

tun. ups 

iauto.iech 

E-6 


■ix-te 


EXPERT-EASE  Ruie  Listing,  Probiet:  AUTOH.RPR  Bate:  25-feb-85 
ieveil 

saai.rpr  :  levei2 

tun. ups  :  eg. kind 

nv.ioo.eg  :  nv.i.eg.i 
autos  :  iauto.iecn 
no.tunup  :  Hob.eq.sv 
no.rpr  .-  iiob.eg.sv 


EXPERT-EASE  Attribute  Listing,  Probiei:  HOB.EQ.SV  Date:  25-feb-85 

type.veh  :  What  kind  of  vehicles  will  this  eipioyee  work  with? 

nor# ai  :  Passenger  cars  and  trucks  (to  include  sens)  that  light,  for  ample, 

stop  at  a  hiway  service  station. 

heavy. eg  :  In  addition  to,  or  instead  of  the  above,  heavy  »obile  equipient  sucn 

as  tanks,  cranes,  tractors,  road  graders  ana  loaders. 

ext.of.svc  :  What  is  the  extent  of  service  this  eipioyee  will  per for*? 

puip.gas  :  Pump  gas,  put  water  in  radiator,  and  clean  windshields  ONLY . 

2greater  :  Provide  lore  service,  such  as  cheat  oil  levels  and  battery  fluid 

levels. 

lub.tir.cn  :  will  this  eipioyee  be  expected  to  do  such  things  as:  repair  tires,  change 
oii,  use  a  battery  tester,  check,  replace  ana  adjust  such  things  as  spark 
?iugs,  fan  belts,  tail  pipes  and  hoses? 
yes  Yes,  lust  nave  the  Knowledge  to  do  these  things, 

no  :  No.  This  kind  of  Knowledge  wiil  not  be  utilized  in  this  job. 

re  pair  :  Will  this  eipioyee  repair  venicies  by  checking  the  condition  of  and 
replacing  any  of  the  following  parts: 

Worn  brake  shoes,  leaky  wheel  cylinders,  corroded  luffiers.  fouied  spar k 
piugs? 

Yes.  : 

NO.  : 

class  :  The  classification  of  this  job  is:  Nobile  Eguipient  Servicer. 

WG-5306-01  : 

*6-5306-03  : 

W6-5306-05  : 

WG-5806-06  : 
lauto.iech  : 
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EXPERT-EASE  Exaipie  Listing,  Problei:  MOB.EQ.SV  Date:  25-feb-85 


type.ven  ext.of.svc  iub.tir.ch  repair 


logical 

logical 

logical 

iogical 

i 

noraal 

puap.gas 

no 

NO. 

♦1 

C 

neavy. ea 

puap.gas 

no 

NO. 

3 

noraal 

greater 

no 

NO. 

4 

noraai 

greater 

yes 

NO. 

5 

heavy. eg 

greater 

no 

NO. 

6 

heavy. eq 

greater 

yes 

NO. 

7 

noraal 

greater 

no 

Yes. 

8 

noraal 

greater 

yes 

Yes. 

0 

heavy. eq 

greater 

no 

Yes. 

0 

heavy. eg 

greater 

yes 

Yes. 

class 

logical 


MS-5306-01 

Mfi-5806-01 

MG-5806-03 

HS-5806-05 

86-5806-03 

M-S806-06 

lauto.iech 

tauto.aech 

lauto.iech 

tauto.aech 


EXPERT-EASE  Ruie  Listing,  Problem  HOB.EO.SV 


Date:  25-feb-85 


repair 

res.  :  lauto.iecn 
NO.  :  ext. of. svc 
puap.gas  :  KS-5806-01 
greater  :  iub.tir.ch 

yes  :  type. veh 

noraai  :  5*6-5806-05 
heavy. eq  :  aG-5806-06 
no  :  W6 -5806*03 
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EXPERT-EASE  Attribute  Listing,  Problei:  AUTO.NECH  Date:  25-feb-85 


aor.tun.up  :  Will  the  eeployee  need  to  have  the  knowledge  to  do  tore  difficult  tasks 
than  engine  tune-ups? 


test.eqip  :  Will  the  etplovee  have  to  be  thoroughly  knowledgeable  in  the  use  of  ALL  of 
the  following  equipment  in  doing  the  job? 


Dynaeoaeter  Osciliiscope  Test  Stnds  Compression  Guage 

Exhaust  Analyzer  Bench  Lathe  Srinding  and  Seating  Equipment 


aa.'.epr  :  Will  the  eapioyee  need  to  be  able  to  tear  down,  repair  and  rebuild  any  !or 
all)  of  the  following  major  parts  of  an  autoiobiie? 


Engine  Transiission  Clutch  Differential  Carburator  Suspension 


close. supv  :  as  this  eapioyee  does  his/her  joo,  will  soaeone  who  knows  how  to  do  tne 
job  check  CLOSELY  on  progress,  and  insure  it  is  being  done  correctly? 
fes.  : 

Ho.  - 


Class  :  The  classification  of  this  position  is: 
w-8  :  Automotive  Worker,  WG-5823-08 

w-10  :  Automotive  Mechanic,  WG-5323-10 

j-b  :  Mobile  Equipment  Servicer,  WG-5806-06 


■  ‘-v 


'  *  * ,»  V  V 
/“in  ’ 


Mr. tun. up  test.eqip  iaj.repr  close. supv  class 
logical  logical  logical  logical  logic; 


i  No. 

t 

1 

Yes. 

u-6 

2  No. 

t 

1 

No. 

ti-8 

3  res. 

Yes. 

1 

NO. 

u-10 

4  Yes. 

Yes. 

* 

Yes. 

«-8 

5  res. 

No. 

Yes. 

No. 

a- 10 

o  Yes. 

NO. 

Yes. 

Yes. 

a-8 

7  Yes. 

No. 

NO. 

t  . 

a-8 

EXPERT-£AS£  Rule  Listing,  Probiea:  AUTO.HECH 


Date:  25-feb-85 


aor . tun. up 
Yes. 


:  ciose.suov 
res.  :  a-3 
Ho.  :  test.eoip 
Yes.  :  u-10 

No.  :  aaj. 


aaj.repr 
Yes.  :  n-10 
Mo.  :  «-3 


:  ciose.supv 
Yes.  :  u-b 
Ho.  :  a-8 


;v_ 

k  - 


■ '  a  -o ;  -a  ii'i :  •;/ 


EXPERT-EASE  Attribute  Listing,  Problei:  FOOD. PREP  Date:  25-feb-85 


griii.ieat  :  Without  detailed  instructions,  does  the  eiployee  assist  in  grilling  chops, 
steaxs,  poultry,  and/or  fish  by  Matching  while  the  iteis  are  cooning, 
turning  as  required,  and  reioving  when  done? 

fes.  : 

No.  : 

ca. goods  :  Without  detailed  instructions,  does  the  eipioyee  open  and  neat  canned 
vegetables  or  fruits?  n 

res. 

No.  : 

gnii.brek':  Without  detailed  instructions,  does  the  eiployee  grill  pancakes,  bacon, 
and  eggs  to  order? 

is.  : 

No.  : 

aeas.mgrd  :  Without  detailed  instructions,  does  the  eepioyee  aea sure  and  weigh 
portions  and  ingredients  as  required  by  recepe,  forsuia  or  diet? 

fes.  : 

No.  : 

ccox.ceri  :  Without  detailed  instructions,  does  the  eiployee  cook  cereal,  beverages 
toast,  salads,  gelatin  and  sandwiches? 

*es.  : 

No.  T,  : 

-iaSS  : 

kCCOii 

Wfd.SV.MKr  : 


EXPERT-EASE  Eiawie  Listing,  Problet:  FOOD. PREP  Date:  25-feb-85 


grill. Mat  can. goods  griil.brek  aeas.ingra  cook.cerl  class 
logical  logical  logical  logical  logical  logical 

1  res. 

res. 

res. 

Y  es. 

Yes. 

icook 

2  NO. 

res. 

Yes. 

Yes. 

Yes. 

Icook 

3  res. 

NO. 

Yes. 

Yes. 

Yes. 

tcook 

4  res. 

res. 

No. 

Yes. 

Yes. 

icook 

5  res. 

res. 

Yes. 

NO. 

Yes. 

icook 

o  Yes. 

Yes. 

Yes. 

Yes. 

No. 

K«jk 

7  t 

i 

« 

NO. 

No. 

ifd.sv.nkr 

8  * 

* 

NO. 

NO. 

* 

ifd.sv.wkr 

9  * 

No. 

NO. 

t 

*  * 

ifd.sv.nkr 

10  No. 

No. 

4 

t 

I 

ifd.sv.wkr 

ii  NO. 

* 

» 

t 

No. 

ifd.sv.wkr 

12  NO. 

» 

* 

NO. 

* 

ifd.sv.wkr 

13  No. 

» 

NO. 

« 

» 

tfd.sv.nkr 

14  l 

NO. 

t 

No. 

ifd.sv.wkr 

15  * 

No. 

t 

NO. 

» 

ifd.sv.wkr 

la  * 

t 

No. 

t 

NO. 

ifd.sv.wkr 

EXPERT-EASE  Ruie  Listing,  P.roble«:  FOOD. PREP 


Date:  25-feb-85 


grill. teat 

Yes.  :  can. goods 

res.  :  grill. brek 

res.  :  seas.ingrd 
Yes.  :  Icook 
No.  :  cook.cerl 
Yes.  :  Icook 
No.  :  lfd.sv.nkr 
No.  :  aeas.ingrd 
Yes.  :  cook.cerl 
Yes.  :  Icook 
Noo.  :  Ifd.sv.nkr 
No.  :  Ifd.sv.nkr 
No.  :  grill. brek 

res.  :  teas.mgrd 

Yes.  :  cook.cerl 
Yes.  :  Icook 
No.  :  Ifd.sv.nkr 
N  o.  :  lfd.sv.akr 
No.  :  lfd.SV.Hkr 
no.  :  can. goods 

Yes.  :  griil.brek 

res.  :  eeas.ingrd 

Yes.  :  cook.cerl 
Yes.  :  Icook 
No.  :  Ifd.sv.nkr 
cNo.  :  Ifd.sv.nkr 
No.  :  Ifd.sv.nkr 
NO.  :  Ifd.sv.nkr 
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EXPERT-EASE  Attribute  Listing,  Probiea:  FD.SV.WKR  Date:  25-feb-85 

nosp.diet  :  Does  tne  eapioyee  work  in  a  hospital  or  other  patient  care  facility  ana 
ao  food  service  work  related  to  the  diets  of  patients? 
res.  : 

‘<0. 

mo*. diet]  :  Does  the  eepioyee  need  to  know  what  types  of  foods  to  serve  on  the  tost 
coaaon  diets  usch  as  soft,  bland,  liquid,  low  caloric,  iow  soaiua,  and 
diabetic? 
res.  : 

MO. 

know.diet2  :  Mill  the  eapioyee  need  to  have  enough  knowledge  about  these  coaaon  aodif- 

ied  diets  to  be  abie  to  answer  patient  questions  about  substitutions,  or 
oto  check  a  diet  tray  for  coapieteness? 
res.  : 

MO. 

^oaou.sai  :  Does  the  eapioyee  need  to  know  how  to  follow  recipes  and  ccaoine  ingred¬ 
ients  for  coaplex  salads  and  other  uncooked  food  products? 
res.  : 

NO.  : 

amor  .cox  :  Whicn  of  the  following  applies  to  this  eaployee's  joo? 

recipes  :  Measures  flour,  sugar,  spices,  and  seasonings  according  to  coaotex 

recipes  provided  by  the  cooxe  or  baker. 

grill  Doing  a  iiaited  variety  of  cooking,  such  as  grilling  or  frying 

sausage,  bacon,  pancakes,  ana  eggs  to  order, 
neither  Neither  of  the  above. 

■■aavy.uk  :  Choose  one  of  the  following: 

fifty. Ibs  :  Eapioyee  wiii  frequently  love  oojects  weighing  over  50  pounds, 

.arge.pans  :  Eapioyee  wiii  do  other  heavy  work  such  as  scouring  and  scrubbing 

neavier  cooking  utensils,  sots  and  pans,  whicn  because  or  their  iar 
site  are  unwieldy  and  awkward  to  handle. 

neitner  Neither  of  the  heavy  work  situations  above  3re  required  of  tne  essi 

oyee. 

class  :  The  classification  of  this  position  is:  Food  Service  Worker, 


w-l 

:  WG-7403-01 

w-2 

:  WG-7408-02 

w-3 

:  MS -7403*03 

w-4 

:  WG-7403-04 
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EXPERT-EASE  Exaiple  listing,  Probiei:  FD.5V.HiCR  Date:  25-feb-3S 


nosp.diec  kncw.dietl  know. diet2  coiph.sal  iinor.cok 
logical  logical  logical  iogical  logical 

heavy. Nk 
iogical 

class 

logical 

1 

res. 

res. 

res. 

4 

4 

4 

14-4 

£ 

res. 

res. 

No. 

res. 

recipes 

4 

1*4 

3 

res. 

res. 

No. 

res. 

grill 

4 

«-4 

4 

res. 

res. 

NO. 

res. 

neither 

4 

k-3 

5 

res. 

NO. 

4 

res. 

recipes 

4 

W"4 

6 

res. 

NO. 

( 

Yes. 

grill 

4 

W-4 

7 

res. 

No. 

4 

Yes. 

neither 

4 

h-3 

3 

Yes. 

No. 

4 

No. 

4 

fifty. lbs 

m-2 

3 

res. 

NO. 

4 

No. 

4 

large. pans  a-2 

.0 

res. 

NO. 

4 

No. 

4 

neither 

M-l 

,i 

NO. 

t 

4 

Yes. 

recipes 

4 

'4-4 

.  4. 

NO. 

t 

4 

Yes. 

grill 

4 

4-4 

i  ** 

LO 

No. 

4 

4 

res. 

neither 

4 

4-3 

.4 

NO. 

4 

4 

No. 

4 

fifty. lbs 

4-2 

l5 

NO. 

4 

4 

No. 

4 

iarge.pans  4-2 

.0 

NO. 

4 

4 

NO. 

4 

neither 

a-i 

EXPERT-EASE  Rule  Listing,  Probiee:  FD.SV.NXR 


Date:  25-feb-85 


csipix.sai 

res.  :  imor.cok 
recipes  :  a-4 
grili  :  k-4 
neither  :  nosp.diet 

Yes.  :  knoa.dietl 

Yes.  :  know.diet2 
res.  :  h-4 
No.  :  »-3 
No.  :  «-3 
No.  :  a-3 
No.  :  heavy. uk 
fifty. ibs  :  frosp.diet 

Ye s.  :  knou.dietl 
Yes.  :  «-4 
No.  :  a-2 
No.  :  a*2 
large. pans  :  hosp.diet 

res.  :  know.dietl 
Yes.  :  a*4 
No.  :  a- 2 
NO.  :  »-2 
neither  :  nosp.diet 

Yes.  :  knoa.dietl 


EXPERT-EASE  Attribute  Listing,  Problei:  COOK 


Date:  2S-feo-35 


supv.know  :  This  eipioyee  works  for: 

<aou  :  A  nigner  graded  cook  or  supervisor  wno  knows  now  to  so  this 

eipioyee’s  job. 

not . xnow  A  supervisor  who  does  not  know  how  to  do  this  eipioyee’s  job. 

irg.variet  :  Does  this  eipioyee  cook  a  FULL  variety  of  foods  inclusive  of  ALL  of  the 
following: 

heats  of  all  kings  Poultry  Seafood  Vegetables  of  ail  kinds 
Nunerous  Sauces  Gravies 

les .  : 

NO.  : 

sany. steps  :  Does  tne  employee  need  to  know  how  to  prepare  lenu  ness  using  special 
or  difficult  recipes  involving  lany  steps,  or  iong  preparation  tne 
INCLUDING  but  not  iiaited  to  spaghetti  sauce  ana  sweet  and  sour  oork. 

iss. 

NO.  : 

on. rise  :  Does  tne  o*pioyee  need  to  be  skilled  in  the  planning,  coordinating  ana 
tiie  seguencmg  of  steps  in  order  to  have  a  variety  of  lenu  1  teas  oeing 
prepared  snultaneousiy  ready  for  serving  at  the  sa»e  tiae  without  over¬ 
cooking  or  waste? 
res.  : 

No.  : 

aoj. recipe  :  Does  this  eipioyee  need  to  appiy  knowledge  and  anility  in  adjusting 
recipes  upward  or  downwar  in  order  to  accoiiodate.a  different  nuiDer 
of  servings  than  called  for  in  the  recipe7 

Yes.  : 

NO.  d  : 


i.ose.supv  :  uces  tn is  esoioyee  work  under  close  supervision  of  another,  wno  is  ncri- 
aiiy  available  to  answer  questions  aoout  cooking,  ana  give  directions  to 
this  eipiovee  in  tne  preparation  of  food7 


c.ass  :  This  position  is  classified  as.- 
w-5  :  Cook.  WG-7ACi5-05 

w-5  :  Cook,  WG-7404-03 
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:Xr£RT-£ASE  Exaiple  Listing,  Probiei:  COOK 


Date:  25-feb-85 


supv.xnow 

logical 

irg.variet  eany. steps  on.tiie 
logical  logical  logical 

adj. recipe  ciose.supv  class 
logical  logical  logical 

know 

res. 

Yes. 

res. 

Yes. 

res. 

w-5 

*> 

Know 

res. 

Yes. 

res. 

Yes. 

No. 

w-8 

3 

know 

res. 

Yes. 

Yes. 

No. 

Yes. 

w-5 

4 

Know 

res. 

Yes. 

Yes. 

No. 

NO. 

w-3 

5 

know 

res. 

Yes. 

So. 

Yes. 

Yes. 

w-5 

b 

know 

res. 

Yes. 

NO. 

Yes. 

NO. 

w-8 

7 

know 

res. 

res. 

NO. 

No. 

Yes. 

w-5 

8 

know 

res. 

Yes. 

so. 

No. 

NO. 

w-5 

9 

know 

res. 

So. 

Yes. 

Yes. 

Yes. 

'w-5 

10 

know 

res. 

No. 

res. 

Yes. 

No. 

w-3 

ii 

know 

res. 

No. 

res. 

No. 

Yes. 

w-5 

12 

Know 

res. 

No. 

res. 

No. 

NO. 

w-5 

13 

knew 

res. 

No. 

MO. 

Yes. 

res. 

w-5 

14 

know 

res. 

No. 

NO. 

Yes. 

No. 

w-5 

15 

know 

res. 

NO. 

NO. 

NO. 

res. 

w-5 

lb 

knew 

res. 

No. 

NO. 

No. 

NO. 

w-5 

17 

know 

Mo. 

Yes. 

res. 

Yes. 

Yes. 

w-5 

13 

know 

So. 

Yes. 

res. 

Yes. 

No. 

w-8 

19 

know 

so. 

res. 

res. 

No. 

Yes. 

w-5 

20 

know 

So. 

Yes. 

Yes. 

No. 

NO. 

w-5 

■'ll 

Cl 

know 

So. 

Yes. 

NO. 

Yes. 

Yes. 

w-5 

22 

know 

So. 

Yes. 

NO. 

Yes. 

NO. 

w-5 

23 

know 

So. 

Yes. 

NO. 

No. 

Yes. 

w-5 

24 

know 

No. 

Yes. 

NO. 

No. 

NO. 

w-5 

25 

know 

No. 

NO. 

res. 

Yes. 

Yes. 

w-5 

2b 

know 

No. 

No. 

Yes. 

.Yes. 

NO. 

w-5 

■}7 

<-  ' 

know 

So. 

No. 

res. 

No. 

res. 

w-5 

23 

know 

NO. 

So. 

res. 

No. 

No. 

w-5 

29 

know 

NO. 

NO. 

NO. 

Yes. 

Yes. 

w-5 

30 

know 

SO. 

NO. 

NO. 

Yes. 

NO. 

w-5 

J  A 

know 

SO. 

NO. 

No. 

No. 

res. 

w-5 

'*'1 

knew 

NO. 

No. 

No. 

No. 

No. 

w-5 

ww 

net. know 

res. 

Yes. 

res. 

Yes. 

Yes. 

w-5 

34 

not. know 

res. 

res. 

Yes. 

res. 

No. 

w-8 

35 

not. knew 

res. 

Yes. 

res. 

No. 

res. 

w-5 

3b 

not. know 

res. 

res. 

res. 

No. 

NO. 

w-8 

j. 

not. know 

res. 

res. 

No. 

Yes. 

res. 

w-5 

33 

not. know 

res. 

Yes. 

NO. 

Yes. 

NO. 

w-3 

39 

not. know 

res. 

Yes. 

NO. 

NO. 

Yes. 

w-5 

40 

not. know 

res. 

res. 

NO. 

NO. 

No. 

w-3 

41 

not. know 

res. 

No. 

res. 

res. 

res. 

w-5 

42 

not. know 

res. 

NO. 

res. 

Yes. 

NO. 

w-8 

43 

not. know 

res. 

NO. 

res. 

NO. 

Yes. 

w-5 

44 

not. know 

res. 

NO. 

res. 

NO. 

No. 

w-8 

45 

not. know 

Yes. 

NO. 

NO. 

Yes. 

Yes. 

w-5 

4b 

not. know 

res. 

No. 

NO. 

Yes. 

NO. 

w-8 

47 

not. know 

res. 

No. 

NO. 

No. 

res. 

w-5 

48 

not. know 

res. 

No. 

NO. 

No. 

No. 

w-3 

49 

not. know 

So. 

Yes. 

res. 

res. 

res. 

w-5 

50 

not. know 

No. 

Yes. 

res. 

res. 

NO. 

w-8 

51 

not. know 

No. 

res. 

Yes. 

No. 

Yes. 

w-5 

52 

not. know 

No. 

Yes. 

res. 

No. 

NO. 

w-3 

53 

not. knew 

NO. 

Yes. 

NO. 

res. 

res. 

w-5 

54 

not. know 

No. 

res. 

NO. 

Yes. 

NO. 

w-8 

55 

not. know 

NO. 

Yes. 

NO. 

No. 

res. 

w-5 

E-2i 
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EXPERT-EASE  Rule  Listing,  Probiei:  COOX 


Date:  25-feb-85 


ciose.supv 

Yes.  :  u-5 
No.  :  supv.knou 

knot!  :  irg.variet 

tes.  :  aany. steps 
Yes.  :  on.ti«e 
Yes.  :  u-3 
No.  :  ad j. recipe 
Yes.  :  u-3 
No.  :  u-5 
No.  :  on.tiae 

Yes.  :  adj. recipe 
N  Yes.  :  u-3 
NO.  :  U-5 
NO.  :  U-5 
No.  :  iany. steps 
Yes.  :  on.tiae 

Yes.  :  adj. recipe 
Yes.  :  u-9 
i  No.  :  u-5 
No.  :  u-5 
No.  :  u-5 

not.knou  :  irg.variet 
Yes.  :  u-3 
No.  :  aany. steps 
Yes.  :  u-8 
No.  :  on.tiae 


Yes.  :  u-8 
No  .  :  adj. recipe 


EXPERT-EASE  Attribute  Listing,  Probie*:  CRANE. OP  Date:  25-feb*85 


crane. typ  :  Two  types  of  cranes  will  be  referred  to,  bridge  and  boo*.  As  used  here 
bridge  cranes  refer  to  those  cranes  which  have  no  boots  and  a  constant 
taxitut  lifting  capacity.  Boot  cranes  refer  to  tnose  cranes  which  nave 
boots  and  varying  taxitut  lifting  capacities  depending  upon  the  Lengtn, 
angie,  and  position  of  the  boot. 

What  kind  of  crane  does  this  etployee  operate? 

8ridge  : 

Boot  : 

brid.confd  :  Mill  the  bridge  cranebe  used  to  tove  loads  over  and  into  production  areas 
?  containing  eguiptent,  tachinery,  supplies,  and  personnel? 

res.  : 

NO.  : 

sent. till  :  Mat  percent  of  the  etpiovee’s  tite  wiii  the  bridge  crane  oe  so  operated0 
(enter  a  nutber  for  tne  percent  of  tite,  e.g.  30  for  thirty  percent): 


ooct.confd  :  Will  tne  ooot  crane  be  operated  in  any  of  the  following  situations? 

Load  and  unload  eguiptent  and  supplies  on  board  ships. 

Precision  set  objects  such  as  gus  into  founts,  sonar  eguiptent  on  to 
snips,  or  wings  on  to  aircraft. 

n  Dig  and  tove  earth  or  rock  close  to  buildings,  walls,  or  underground 
destructions. 

Destroy  brick  and  stone  structures  near  nign-voltage  power  lines  or 
e  otner  structures. 

res.  : 

No. 

pcnt.tit2  :  what  percent  of  the  etpiovee’s  tne  will  the  boot  crane  be  so  operated7 
renter  a  nutoer  for  the  percent  of  tite,  e.g.  50  for  thirty  percent): 


I'joi.cao  :  win  tne  ooot  crane  oe  operated  when  tne  crane  is  at  or  near  its  lifting 
capacity' 
res. 
vc. 

ptnt.tiio  .-  wnat  percent  of  the  eipiovee's  cue  will  the  ooot  crane  oe  so  operated? 

renter  a  nueoer  for  tne  percent  of  tne,  e.g.  30  for  thirty  percent): 


:  The  classification  of  this  position  is:  Crane  Operator, 
W-T  :  WG-5725-07 

w-N  :  WG-5725-09 

w-il  :  '46*5725*11 


'lass 


EXPERT-EASE  £xa*ple  Listing,  Profile*:  CRANE.OP  Date:  25-feb-85 


erane.typ 

firid.confd  ocnt.ti*! 

booa.confd  pent . tia2 

boo*. cap 

pcnt.tiaS 

class 

logical 

logical 

integer 

logical 

integer 

iogical 

integer 

iogical 

1 

Bridge 

Mo. 

i 

4 

4 

4 

4 

w-7 

a 

L 

Bridge 

Yes. 

24 

4 

4 

4 

4 

4-7 

3 

Bridge 

Yes. 

25 

4 

4 

4 

4 

4-9 

4 

Boo* 

t 

4 

NO. 

4 

NO. 

4 

4-9 

5 

Boo* 

t 

4 

Yes. 

24 

NO. 

4 

4-9 

6 

Boo* 

% 

4 

Yes. 

25 

NO. 

4 

4-il 

7 

Boo* 

i 

4 

NO. 

4 

Yes. 

24 

a-9 

3 

Boo* 

» 

4 

NO. 

4 

Yes. 

25 

4‘11 

9 

Boo* 

i 

4 

Yes. 

24 

Yes. 

24 

4-9 

10 

8oo* 

% 

4 

Yes. 

25 

Yes. 

24 

4-11 

11 

Boo* 

* 

4 

Yes. 

24 

res. 

25 

E-25 
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EXPERT-EASE  Rule  Listing,  Probiei:  CRANE. OP  Date:  25-feb-35 
crane. typ 

Bridge  :  brid.confd 

res.  pent. till 
<25  :  '4-7 
>=25  :  «-9 
No.  :  4-7 
Booi  :  booi. cant'd 

Ye.s.  :  pent. tin2 

<25  :  booi. cap 

Yes.  :  pcnt.tii3 
<25  :  4-9 
>  =25  :  4-11 
NO.  :  4-9 
>=2S  :  4-11 
No.  :  boo*. cap 

Yes.  :  pcnt.tiio 
\25  :  4-9 
>=25  :  4-li 
No.  :  4-9 


EXPERT-EASE  Attribute  Listing,  Probiei:  MO.VEH.GP  Date:  25-feb-35 

i. veil. opr  :  This  eipioyee  wiii  operate  lotor  vehicles.  Are  AMY  of  the  following 
vehicles  operated  by  this  eipioyee? 

Tanks  or  other  tracked  vehicles  3uiioozers,  graders  and  otner  eartn 

aoving  and  construction  eouiDient 

Warehouse,  industrial  and  fan 

tractor  Road  sweepers,  aircraft  refueiig 

vehicles,  crash  trucks,  ano  air- 
Forklift  trucks.  craft  toning  vehicles, 

res. 

.nn  ..  No.  : 

seii.truk  :  Is  the  vehicle  operated  a  "tractor-trailer  type"  truck  utilizing  a  fifth 
wheei  or  pivot  for  tne  trailer'’ 
res.  : 

No.  : 

ous.oor  :  Is  the  vehicle  operated  by  this  eipioyee  a  bus? 
res.  : 

NO.  : 

dus. size  :  what  is  the  passenger  capacity  of  the  bus  (enter  a  nuiber  for  the  taxiaua 
er  of  passengan  the  bus  is  designed  to  accoaiodate,  e.g.  15): 


tyo.spec.v  :  What  type  of  power  takeoff  equipient  is  on  the  vehicle: 

wench  :  Wench  for  loading  and  unloading  heavy  aateriais.  I 

trash  :  Trash  container  lifter  and  trasn  coioactor  BOTH  OPERATED  BY  THIS 

EMPLOYEE. 

otner  :  '  ‘Cherry  picker"  type  equipient  OPERATED  BY  THIS  EMPLOYEE. 

soec.ven  :  Is  the  vehicle  operated  a  special  purpose  vehicle,  i.e.  one  witn  a  power  : 

takeoff  to  aower  special  ourpose  equipient  on  the  tmcw? 

,'es.  : 

NO.  : 

gvw  :  Enter  the  gross  vehicle  weight  of  the  vemcie  operated  (enter  a  luaper  m 
POUNDS,  e.g.  5000  for  5000  Bounds/: 


;;ass  :  The  classification  of  this  position  is:  Motor  Vehicle  Operator, 


w-5 

:  W6-5703-05 

w-o 

:  WG-5703-06 

w-7 

:  WG-5703-Q7 

w-3 

:  WG-5703-03 

IMAGE. GRAD 


E-27 


EXPERT-EASE  Eiaipie  Listing,  Problem  NO.VEH.OP 


e.ven.apr 

seei.truk 

bus. opr 

bus. size 

logical 

logical 

logical 

integer 

i 

<es. 

t 

i 

t 

2 

No. 

res. 

f 

i 

5 

No. 

NO. 

Yes. 

20 

4 

NO. 

No. 

Yes. 

21 

5 

NO. 

No. 

No. 

4 

NO. 

No. 

No. 

4 

7 

NO. 

NO. 

NO. 

4 

8 

NO. 

No. 

No. 

4 

4 

NO. 

No. 

No. 

4 

10 

NO. 

No. 

No. 

4 

11 

No. 

NO. 

No. 

* 

12 

No. 

No. 

NO. 

4 

13 

NO. 

NO. 

No. 

4 

14 

No. 

NO. 

No. 

4 

K 

No. 

No. 

NO. 

4 

Is 

NO. 

NO. 

No. 

4 

.  ? 

NO. 

NO. 

NO. 

4 

Date:  25-feb-85 


typ.spec.v 

spec. veh 

gvu 

class 

logical 

logical 

integer 

logical 

4 

4 

4 

tWA6£ . BRAD 

4 

4 

4 

w-3 

4 

NO. 

4 

w-8 

4 

No. 

4 

w-7 

wench 

Yes. 

7000 

w-s 

trash 

Yes. 

7000 

w-7 

other 

Yes. 

7000 

w-7 

4 

No. 

7000 

w-5 

4 

No. 

7001 

w-6 

4 

No. 

21989 

w-s 

4 

4 

22000 

W-7 

wencn 

Yes. 

7001 

W-5 

trash 

Yes. 

7001 

w-7 

other 

Yes. 

7001 

w-7 

wench 

Yes. 

219*9 

w-b 

trasn 

Yes. 

it  090 

w-7 

other 

res. 

21999 

w-7 

E-28 


EXPERT-EASE  Rule  Listing,  Problem  NO.VEH.OP 


Date:  25-feb*8S 


a. yen. opr 
res. 

NO. 


:  WAGE.  SHAD 
:  seai.tr uk 
res.  :  a-8 
No.  :  gva 

<7001  :  bus. opr 

Yes.  :  bus. size 

<21  :  N-6 

>=21  :  a-7 
No.  :  spec.veh 

Yes.  :  typ.spec.v 
.  aencii  :  a-6 
trash  :  a-7 
other  :  a-7 
No.  :  a-5 

>=7001  :  gva 

<22000  :  spec.veh 

Yes.  :  typ.spec.v 
aench  :  a-b 
trash  :  a-7 
otner  :  a-7 
No.  :  bus. opr 

Yes.  :  bus. size 

<21  :  w-o 
>  =21  :  a-7 
No.  :  a-b 
) =22000  :  bus. opr 

Yes.  :  bus. size 
<21  :  a-b 
>=21  :  a-7 
No.  :  a-7 


E-29 
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EXPERT-EASE  Attribute  Listing,  Proble*:  FILLET. OP  Date:  25-feb-35 


ioad.lft  :  Enter  the  load  lifting  capacity  of  the  fork  lift  this  eipioyee  will 
operate  (enter  as  a  nuaoer  of  POUNDS,  e.g.  3000  for  3000  pounds): 


outside  :  Aiii  the  eapioyee  operate  the  forklift  outside? 
res.  : 

No.  : 


rough . ter  :  Is  the  outside  terrain  ROUSH,  or  relatively  LEVEL? 

Rougn 

Levei 

regular  :  Is  it  a  CERTAINTY  that  outside  operation  over  Rough  Terrain  aill  occur  AT 
LEAST  once  in  the  next  lonth? 

Tes.  : 

No.  : 

'scoring  :  Nil!  operation  of  the  forklift  in  rougn  terrain  occur  at  scneiuiec  regular 
intervals  such  as  every  day,  every  Friday,  or  other  predicted  intervals? 
"es.  : 

NO.  : 


.ift.cap  :  Hhat  is  tne  percent  of  tue  the  eepioyee  will  operate  this  fornlift  to 

lift  loads  of  10,000  pounds  or  • ore?  (enter  as  a  nutoer.  e.g.  15  for  15I; 


class  :  The  classification  of  this  position  is:  Fork  Lift  operator, 
«-5  :  MG-5704-05 

«i-s  :  NG-57G4-06 


E-30 
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EXPERT-EASE  Exaepie  Listing,  Probiee:  F1.LFT.0P  Date:  25-r'eb-35 


ioad.ift 

outside 

rough. ter 

regular 

recuring 

lift. cap 

class 

integer 

logical 

logical 

logical 

logical 

integer 

logical 

'599 

Ho. 

i 

<9 

* 

f 

■4-5 

•> 

9999 

res. 

Levei 

♦ 

* 

* 

4-5 

3 

9999 

Yes.  • 

Rough 

HO. 

* 

* 

•4-5 

9999 

Yes. 

Rough 

Yes. 

No. 

1 

4-5 

5 

9999 

Yes. 

Rough 

Yes. 

Yes. 

» 

W“0 

S 

10000 

Ho. 

» 

* 

* 

24 

4-5 

7 

10000 

Yes. 

Level 

* 

* 

24 

4-5 

3 

10000 

Yes. 

Rough 

* 

t 

t 

4-6 

9 

10000 

No. 

* 

o  * 

t 

25 

4-6 

10 

10000 

Yes. 

Levei 

* 

t 

25 

4-b 

E-31 
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EXPERT-EASE  Rule  Listing,  Probiet:  FK.LFT.OP 


Date:  25-feb-85 


ioaa.ift 

(10000  :  outside 

Yes.  :  rougn.ter 
Rougn  :  reguiar 

tes.  :  recuring 
Ye s.  :  k-6 
Mo.  :  4-5 
Mo.  :  4*5 
Level  :  4-5 
No.  :  4-5 
>  =  10000.  :  lift. cap 

125  :  outside 

Yes.  :  rougn.ter 
Rougn  :  u-o 
Level  :  4-5 
Mo.  :  4-5  ' 

>  =25  :  w -b 


E-32 


EXPERT-EASE  Attribute  Listing,  Proplei:  AIR.C.MEC  Date:  25-feb-35 


<ind.supv  :  Which  of  the  following  lost  neariy  describes  the  kind  of  supervision  tiie 
eioiovee  will  work  under7 

indepeno  :  The  eipioyee  is  expected  to  oe  fully  capable  of  oer forming  ail 

aspects  of  air  conditioningiaintenance  ano  repair  witn  iittie  or  ic 
tecnnicai  supervisin  of  any  kind. 

dose  :  The  eipioyee’s  work  is  usually  accomplished  without  ineoiate 

supervision,  however,  technical  supervision  is  available  when  needed. 

puips  :In  repairing  air  conditioners,  does  tn is  eipioyee  need  to  have  the  k  so  know¬ 
ledge  to  dismantle,  repair,  ano  reassemble  puips? 
res. 

No. 

inceiiers  :  Does  this  eipioyee  need  to  nave  the  knowledge  to  be  abie  to  dismantle, 
repair,  and  reasseioie  a  variety  of  impellers  anc  coipressors7 
res.  : 

NO.  : 

crullers  :  Dees  the  esoioyee  need  to  have  the  knowledge  to  dismantle .  repair,  and 
reasseibie  a  nuioer  of  different  kinds  of  chillers? 

res.  : 

NO. 

receivers  :  Is  the  eipioyee  expected  to  have  the  knowledge  to  accomplish  full  'eoair 
of  receivers' 

res.  : 

No .  : 

evaporator  :  Does  the  eipioyee  need  to  nave  extensive  knowledge  of  the  operation  of 
evaporators  in  order  to  lake  ccipiete  repairs  of  them"7 
res. 

NO. 

•  bds  :  Is  the  einovee  expected  to  nave  the  knowledge  to  perron  complex  repairs 
bv  installing  and  fitting  connecting  rods' 

'ac  ; 

NO.  : 

ernx. snfts  :  Is  a  knowledge  of  installing  and  fitting  crank  shafts  expected  of  tms 
eipiovee  m  doing  the  job7 
res. 

NO. 

Pist. rings  :  Does  the  eioioy  ee  need  to  possess  the  skiii  and  knowledge  sufficient  to 
take  repairs  by  installing  and  fitting  Piston  rings7 
es.  : 

wo. 

searings  :  Is  tne  eioiovee  expected  to  have  the  knowledge  to  be  able  to  lake  'ecair? 

-men  mv Love  the  reolacesent  or  oearings  and  bushings' 

!  oc 

NO. 

/iives  ;  Is  -nowieoge  of  overhauling  valves  expected  of  this  eioiovee ; 
i  es. 

NO. 

::ec.purp  :  will  this  eipioyee  work  with  special  :ur?ose  a: r  conditioning  jnirs  ihd 


r.n 


EXPEST-EASE  Attribute  Listing.  Proole*:  AIR. C. NEC  Date:  25-feb-85  Page: 


systems  tnat  are  used  in  TESTS  or  EXPERIMENTAL  LABORATORIES.  AND  are  used 
to  nave  a  DIRECT  AFFECT  on  tne  tests  or  experiaents? 
res. 

**Q.  : 

class  :  This  position  is  classified  as:  Air  Conditioning  Eauipaent  Mecnanic 


a-8 

:  46-5306-03 

4-10 

:  46-5306-10 

a- 1 1 

:  46-5306-11 

E-  34 


EXPERT-EASE  Rule  Listing,  Probiei:  AIR. C. NEC  Date:  25-fet>-85 

kind.supv 

mdepend  :  ouaps 

Yes.  :  iipeiiers 
Yes.  :  chillers 

Yes.  :  receivers 

Yes.  :  evaporator 
Yes.  :  rods 

Yes.  :  crnk.shfts 

Yes.  :  spec.purp 

Yes.  :  pist. rings 
Yeis.  :  bearings 
Yes.  :  *-11 
No.  :  valves 
Yes.  :  *-11 
NO.  :  *-0 
No.  :  bearings 

Yes.  :  valves 
res.  :  *-11 
No.  :  a-io 
No.  :  valves 
res.  :  *-10 
No.  :  a-3 

a  No.  :  Pist. rings 
Yes.  :  *-10 
No.  :  bearings 
Yes.  :  *-10 
No.  :  vaives 
Yes.  :  *-10 
,  No.  :  a-3 

No.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  vaives 

Yes.  :  spec.purp 
Yes.  :  *-li 
NO.  :  a-iO 
No.  :  *-10 
No.  :  vaives 
Yes.  :  *-10 
No.  :  *-3 
No.  :  bearings 
Yes.  :  vaives 
Yes.  .-  *-10 
No.  :  *-8 
No.  :  *-3 

No.  :  crnk.shfts 

Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  vaives 

Yes.  :  spec.purp 
?es.  :  *-11 
NO.  :  a-iO 
No.  :  *-10 
No.  .-  vaives 
Yes.  :  *-10 
No.  :  a-3 
No.  :  bearings 
res.  :  vaives 


EXPERT-EASE  Ruie  Listing,  Probiei:  AIR.C.MEC  Date:  25-feb-85  Page:  2 

No.  :  »-8 
No.  :  b-3 
No.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  valves 
Yes.  :  a-10 
No.  :  a-8 
NO.  :  h-8 
No.  :  i*-8 

No.  :  rods 

ies.  :  cr nk.shfts 

Yes.  :  oist. rings 
Yes.  :  bearings 
Yes.  :  vaives 

Yes.  :  spec. pure 
Yes.  :  w-i; 
No.  :  »-iC 
No.  :  sa-LG 
No.  :  vaives 
res.  :  N-iO 
No.  :  a-8 
No.  :  bearings 
Yes.  :  vaives 
Yes.  :  n-iG 
No.  :  N-0 
No.  :  H*3 
No.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  vaives 
Yes.  14*10 

NO.  :  14*8 

No.  :  a-3 
No.  :  14*3 
No.  :  cm. softs 

res.  :  pist. rings 
Yes.  :  bearings 
1 8$.  :  .'Oives 

;e$.  :  A-': 3 
NO.  :  14*8 
No.  :  a- 3 
No.  :  a-8 
No.  :  a-8 

No.  :  evaporator 
Yes.  :  roas 

Yes.  :  ernk. shits 

Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  vaives 

Yes.  :  spec.puro 
res.  :  >ri 
NO.  :  4-1 
No.  :  a-iO 
No.  :  vaives 
Yes.  :  a-iO 
No.  :  !4-8 
No.  :  oearings 
res.  :  vaives 


EXPERT-EASE  Ruie  Listing,  Proble*:  AIR.C.HEC 


Date:  25-feb-85 


Page:  3 


So.  :  4- 8 
NO.  :  4-8 
No.  :  Pist. rings 
Yes.  :  bearings 
Yes.  :  valves 
Yes.  :  4*10 
No.  :  4-8 
No.  :  4-8 
No.  :  4-8 
No.  :  crnk.shfts 

Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  valves 
Yes.  :  4-10 
No.  :  4*8 
NO.  :  4-8 
No.  :  4-8 
No.  :  4-3 

No.  :  rods 

Yes.  :  crnk.shfts 

Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  vaives 
Yes.  :  w-lD 
NO.  :  4-8 
No.  :  4-8 
No.  :  4-3 
No.  :  4-8 
NO.  :  4-8 

No.  :  receivers 

Yes.  :  evaporator 
Yes.  :  rods 

Yes.  :  crnk.shfts 

Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  valves 

Yes.  :  spec. pure 
Yes.  :  4-ii 
No.  :  4-10 
NO.  :  4-10 
No.  :  valves 
Yes.  :  4-10 
NO.  :  4-3 

No.  :  bearings 
Yes.  :  vaives 
Yes.  :  «-10 
No.  :  4-3 
No.  :  4-8 
No.  :  oist. rings 
Yes.  :  bearings 
Yes.  :  valves 
'res.  :  4-10 
No.  :  4-3 
NO.  :  4-8 
No.  :  4-8 
No.  :  crnk. snfts 

res.  :  pist. rings 
res.  :  bearings 

E-37 


EXPERT-EASE  Rule  Listing,  Problem  AIR. C. NEC 


Date:  25-feb-85 


Page:  4 


Yes.  :  valves 
res.  :  4-10 
MO.  :  ‘4-8 
Mo.  :  w-8 
Mo.  :  4-8 
Mo.  :  4-3 

NO.  :  rods 

Yes.  :  crnk.shfts 

res.  :  Pist. rings 
res.  :  bearings 
res.  :  vaives 
Yes.  :  4-10 
No.  :  4-3 
Mo.  :  4-3 
No.  :  4-3 
Mo.  :  4-8 
NO.  :  4-3  ' 

Mo.  :  evaporator 
res.  :  rods 

Yes.  :  crnk.shfts 

Yes.  :  Pist. rings 
Yes.  :  bearings 
res.  :  vaives 
Yes.  :  4-10 
No.  :  4-8 
No.  :  4-8 
No.  :  4-3 
Mo.  :  4-8 
NO.  :  4-3 
No.  :  4-8 

No.  :  chillers 

Yes.  :  receivers 

Yes.  :  evaporator 
res.  :  rods 

res.  :  crnk.shfts 

res.  :  pist. rings 
res.  :  bearings 
res.  :  vaives 

Yes.  :  spec.purp 
res.  :  4-1 

No.  :  4*1 
No.  :  4-10 
No.  :  valves 
res.  :  w-iO 
No.  :  4-8 
No.  :  bearings 
Yes.  :  valves 
Yes.  :  4-iO 
Mo.  :  4-8 
No.  :  4-8 


EXPERT-EASE  Rule  Listing,  Proble*:  AIR.C.HEC 


Date:  25-feb-85 


Page:  5 


Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  valves 
Yes.  :  4-10 
No.  :  4-8 
NO.  :  a*8 
NO.  :  4*8 
No.  :  «-3 

No.  :  rods 

Yes.  :  crnk.shfts 

Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  vaives 
Yes.  :  u-10 
NO.  :  a*3 
NO.  :  4*8 
No.  :  4-8 
No.  :  a-8 
No.  :  a-8 
No.  :  evaporator 
Yes.  :  rods 

Yes.  :  crnk.shfts 

Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  valves 
Yes.  :  a- 10 
No.  :  a-8 
No.  :  4-8 
NO.  :  4*8 

*  No.  :  a-8 

No.  :  a-8 
No.  :  a-8 
No.  :  receivers 

Yes.  :  evaporator 
Yes.  :  roas 

Yes.  :  crnk.shfts 

Yes.  :  pist. rings 
Yes.  :  bearings 
Yes.  :  valves 
Yes.  :  a- 10 
No.  :  a-3 
No.  :  a-8 
No.  :  a-3 
No.  :  a-8 
No.  :  a-8 
NO.  :  4-8 
No.  :  4-8 

No.  :  uoeilers 
'es.  :  chillers 

Yes .  :  receivers 

'es.  :  evaporator 
Yes.  :  roos 

Yes.  :  crnk.shfts 

Yes.  :  Pist. rings 

Yes.  :  oearmgs 
?es.  :  vaives 

Yes.  :  spec. Pure 


EXPERT-EASE  Rule  Listing,  Problem  AIR. C. NEC  Date:  25-feb-8S 


No. 

HO.  :  «-8 
NO.  :  «-8 
HO.  :  «-8 
NO.  :  ti-3 
Ho.  :  h-8 
No.  :  chillers 

Yes.  :  receivers 

Yes.  :  evaporator 
Yes.  :  rods 

Yes.  :  crnk.shfts 

Yes.  :  pist. rings 


SOURCE  COSE  FOR  BRIBFING 


I 

This  appendix  contains  the  computer  source  code  for  a  briefing 
prepared  to  explain  expert  systems.  This  briefing  has  been 
presented  to  two  groups  of  personnel  professionals  in  the 
Pentagon  on  two  separate  occasions,  at  a  Western  Area  CPO 
|  conference,  to  a  group  of  personnel  from  the  OPM  Western  Region 

Office,  and  to  other  various  small  groups  and  individuals  on 
numerous  occasions.  It  is  being  provided  in  this  report  since  it 
was  developed  during  the  course  of  the  fellowship  project,  and 
the  programming  techniques  used  may  be  of  interest  to  others. 

1  The  program  is  written  in  BASICA  from  Microsoft.  The  time 

expended  on  writing  it  was  somewhere  between  80  and  120  hours. 
It  employs  the  use  of  animated  presentation  of  words  and  graphics 
and  utilizes  color.  It  has  NOT  been  optimized  for  efficiency  of 
code  (I  wrote  as  quickly  as  possible  without  concern  for 
structure,  code  size  or  efficiency).  It  was  also  my  first 
attempt  at  coding  in  BASICA,  although  I  have  used  other  versions 
of  Microsoft  BASIC  which  do  not  have  many  of  the  graphics 
features  of  BASICA. 

I  would  be  happy  to  discuss  this  technique  for  preparing  and 
presenting  briefings  with  interested  persons. 


1  ’  8RIEF513 

J  » 

3  GOTO  1000 

4  ’ 

5  ’ 

6  ’Briefing  on  Expert  Systems  for  Use  in  Army  Civilian  Personnel  Administration 


10  'Fro#  a  study  conducted  under  a 

11  ’Secretary  of  the  Army  Research  and  Study  Fellowship 

12  ’ 

13  ’ 

14  ’ 

15  ’Program  written  by  Loren  D.  Martindale 

16 

17  ’ 

13  ’ 

19  ’ 

20  ’(c)  Copyright  Department  of  the  Army  and  Loren  D.  Martindale,  I°85 

21  ’ALL  RIGHTS  RESERVED 

22  ’ 

23  ’ 

24  ’ 

25  ’ 

50  BKUFMNKEYMF  BKUP$="'  THEN  50 

51  RETURN 

1000  CLEAR : SCREEN  1:UIDTH  40:KEY  OFF : CLS : DEFINT  A-Z 

1002  ’ . BLOCK  LETTERS  . 

1304  HAJ: "U5E10R3F 10D5L3U5L17D5L3BU7BR6E5R1F5L11 “ 

1006  MCJ- "H2U1 1E2R20F2D2L3U2L 16G2D7F2R16U2R3D2G2L19 ‘ 

1003  "U15R20F2D11G2L20BE2BR1U11R15F2D7G2L15* 

1010  MEJ--'U15R22D3H9D3R11D3L11D3R19D3L22* 

1012  MFJ:  *U15R22D3L1<)D3R1 1D3L11D6L3" 

1014  NI$:'U2R5U11L5U2R13D2L5D11R5D2L13‘ 

1016  MIJ-'U15?.3D13R13D2L20" 

1013  !WJ:‘Ui5R3F9R2E9R3D15L3UIlG9L2H’?DIlL3“ 

1020  Hf!$:'U15R3F13U13R3D15L5HllSllL3* 

1022  N0$:'H2U11E2R13F2D11G2L13BE28R1H2U7E2R12F2D7G2L12“ 

1024  m- "U15R20F2D6G2L 16D5L3BU7BR3R13E2U2H2L13S6” 

1026  JtR J = * U 1 5 R20F2 D5G2L6F 6L3H6L 7D6L36U3BR3R13E2U1H2L130S~ 

1023  MSi--"Rl<I2U3H2L16H2U2E2R14F2R2U2H2L13G2D5F3RiSD3Ll6H2L2D2F2R3“ 
1030  HTtr*U12L3U3R22D3L9012L4* 

1032  "ft-12, -  15R3M+10 , 11M+10, -1 1R3M-12, 151 1 
1034  HY$:'U5H1GR3F3R2E3R3G10D5L3‘ 

103*  DRAW‘clt>al25,20; >aa$;“ 

1033  DRAU”bal55,20;.(int;' 

1040  PAINT! 126, 19), 1: PAINT! 15*,  1°)  ,1 

1042  ’ . LASER  LOGO  'EXPERT  SYSTEMS' . . 

1044  ’ 

104*  A|;  'Hr?.-J!t*2.-1N43.-1R7”*  U  L  CORNER 
10  - •  BJ.-',s:ft+2,liUl,!D2"»  U  S  CORNER 
1050  Ct:'H-I.!H-2.2f!-2,lfl-3,:!1-3,Ili”»  t  R  CORNER 
1052  0J:‘L:N-3,-lN-2.-lff-l.-lUlNn.-i*’t  L  L  CORNER 
1054  “i'"H-l , IL2M-1 , 1L2M+1 , 1R2M+! , 1 " ’ *  CURVE  FOP 


1056  E$=*M+1 , -lRlfH-1 , -IL1M+1 , -1L1M-1 , - LL3H- 1 , -1L3M-1 , -1L3M-2 . -1L2U3’ '*  CURV  S' 

1053  L$- “NM160 , 182C0NHlt-0 , 182C1 "  ’ *  LASER  ORIGIN  POINT 
1060 

LL|: 'NH160 , 182NU4ND5NL10NR10NE4NF5NG5NH4CONM160 , 182NU4ND5NL10NR10NE4NF5NG5NH4C1 “ ’ ** 
SPARKLER  ON  END  OF  LASER 
1062  ’  large  “E‘ 

1064  DRAM  ‘BH351oO;XLL$;R21;XL$;H-7,7;XLS;L17;XL$;M-j,3;XLi;R7;XLJ;M- 

5,5;XLJ;L7;XLt;N-2,2;XL$;R17;XLJ;H-6,6;XL$;L13;XD$;XLt;H+15,- 

15;XL$;XA$:XLt;R20H-717;L17;M-3,3;R7H-515L7M-2,2R17H-6,6LI8;XD$;' 

1066  ’  'X' 

1068  DRAM  "BH70,30;XLL$;R15;XLt;M+l,5;XLt;N+13,'5;XLt;R15;XL$;H' 
26,ll;XLt;M*3I12;XL*;LlS;XLt;H-l,-5;XL*;H-1315;XLi;L15;XL*;M+25,-ll;XLt;«-31- 
12 ;  XL$ ;  316M+1 , 5H+13 , -5R15M-26 , 11M+3, 12L15M-1 , -5M~13 , 5L15M+25 , -11M-3 , -12“ 

1070  ’  *P* 

1072  DRAH‘BH135,30;XLLt;R20;XLJ;XB$;XLi;H-7,7;XLi;XCt;XLt;Lll;XLi;H+5,- 
5;XLt;R7;XLB;NH,-6;XLt;L12;XLt;N-17,17;XL*;L14;XL$;Nt20,-20;XLi;XA$;XLJ;' 

1074  DRAW BM135 , 30R20; XB$ ; M-7 , +7 ; XCJ ; L 1 1M+5 , -5R7M+6, -6L12M-17 , 17L14M+20 , - 
20;XAt;"»«»  REDRAW  UPPER  ’P' 

1076  ’  SMll  T 

1078  DRAM  ■BHlS7,30;XLLJ;R21;XL$;N-7,7;XLt;L17;XLJ;N-3,3;XLt;R7;XLJ;H- 

5,5;XLt;L7;XL$;H-2,2;XL$;R17;XL$;H-6,6;XLB;L18;XDt;XL$;HH5,- 

15;XLt;XA$;XLX;R20H-7,7;L17:H-3,3;R7M-5,5L7M-2,2R17H-6,6L18;XDX;' 

1080  ’V 

1082  DRAH‘BM229>30;XLLt;R20;XL$;XB$;XL$;H-7,7;XL$;XC$;XLt;H+6,6;XL$;L14;XL$;N- 

6,-6;XLX;L2;XLX;N-616;XLt;L14;XL$;H+20,-20;XLJ;XAJ;XLS;BHt0,6;R10;XLJ;N- 

6,6;XLi;L10;XLJ;H*6,-6;XLt; 

1034  DRAH'BH229,30R20;XBt;H'717;XCI;H+6,6;L14;N"6,"6;L2;H'6,6;L14;H+20, • 
20;XAJ;BfH0,6;R10;N-6,6;L10;N+8,-6;' 

1086  ’  'T* 

1038  DRAM  ‘BH270,30;XLLi;R50;XLJ;H-717;XLI;L17;XLt;H-16,16;XLl;L15;XLS;N+16,- 
16;XLt;L18;XLt;H+7,-7;XLJ;R50M-7,7L17H-16,l«LlSHH6,-16Li3H+7p-7* 

10%  ’  large  'S' 

10Q2  DRAM  "3N25,70;XLLX;R36;XLt;l1-6,6;XLt;L16;XL+;XFJ;XL$;R9;XLI;XB$;XL5;N- 
3t3:XLJ;XCJ;XL$;L32:XLt;H+6,-6;XLJ;R19;XL$;XE$;XL$;HM,-«;XLJ;XA$;XLt;R30H- 
n,6Ll6;XF$;R9;XBt;N-3,3;XCi;L32N+6,-6R19;XEl;Nt4,-4;XAi;' 

1094  ’  'Y' 

109b  DRAW  ■3H71,70;XLLt;R15;»L$;N+l,5;XL$;HH3,-5;XLt;R15;XLJ;H-26,il:XLt;H- 

l2.12;XLi;Ll5;XLI;Hni,'I2:XL$;N-2,-ll;XLt;R15HU,SHH3,-5Rl5H-26,llH- 

i:,12L15N+ll,-12N-2,-ll’ 

1098  ’  stall  'S' 

1100  DRAW  'BH129, '0;XLL$;R36;XL$;H-6,6;XL$;L16;XLt;XF$;XLJ;R9;XL$;XBI;XL$;H* 

3,3;XLI;XCJ;XLI;L32;XLt;H*6f-6;XLI;R19:XL$;XEJ;XL$;H*4I-4;XLI;XAJ:XLI;R30H- 

e16L16;XF|;R9;XB!;N-3,3;XCt;L32«+6,-6R19;XEI;M+4,-4;XA|;' 

1102  ’  'T* 

1104  DRAW  'BM175,70;XLLl;R40;Xlt;H-7)7;XL$;L12;XU;H-16.l6:XLi:L15;XLJ;Hri6.- 
16;''LJ;L13;XLS:H+7,-7;XLt;R40H-7,7L12H-16,16L15NM6,-i6L13Hr7,-7' 

1106  ’  stall  “E‘ 

1103  DRAW  *811235, 70;XLL$;R21;XLt;ll-7I7;XL$;L17;XL$;n-3, 3;»LI;R7;XLI;H* 
5t5;XLJ:L’;Xl$;H-2,2:XL$:Rl7;XLt;H-6,6;XLJ;L13:X0$:XL$;«U5,- 
15;ai;<AJ:(LJ;R20N-7,7;L17;.1-313;R7N-5,5L7H-2,2RI7H-6,6ll8;XD$;«M5,-15;XA$;' 


: ; 12  DRAM’SR?75, 70:XLL{;R37;XLI:XBS;XLt;N" 19,19; <LS;L14:XLJ;H*17, - 

r:<Li;L':^Lt:I1-i7,l7;XLt:L14:XLS:HU7.-17;XL$;Nf1-17,l’L7;<L;:M- 
17,1':  :kJ:LI4:-'LJ;*H20,*20;XA$;R35;XEJ;>I-I<»,l'»Li4H+!',-17LTN-I7,!7L!4N«I7 


1114  ’  s»all  'S' 

lilt,  DRAM  ’BH5.100;XLL*;R300;XLL*;L300R300M-6,6;XLL*;H+6,-6ri-6,6L300;XLL*;t1+6. 
6 ;  XLLt ;  6fl5 , 100 ;  RJ00M-6 , 6L300M+6 ,  -6  “ 

1113  PAINT  ( 31 , 41 ) , 1 
1120  PAINT  (72,35) ,1 
1122  PAINT ( 140 , 32) ,  1 
1124  PAINT! 190,321,1 
1126  PAINT ( 235 , 32 ) ,  1 
1128  PAINT(275, 321,1 
1130  PAINT  (30,721,1 

1132  PAINT(75, 721,1 
1134  PAINK132, 72),1 
1136  PAINTU80, 721,1 

1133  PAINT(240, 72),1 
1140  PAINT(276, 721,1 
1142  PAINTI10, 1021, 2,1 
1144  COLOR  14 

1146  LOCATE  15,17:PRINT  “F  0  R" 

1148  DRAW"c2b«35, 150;xaic$;  ‘ 

1150  DRAW“b»70,150;.<li$;" 

1152  DRAW‘b*105,l50;Xiv$;“ 

1154  DRAW“bll30,150;X*it;" 

1156  DRAW"bil55,l50;XiU;" 

1153  DRAM" t>«135, 150 ;  x®i$ ;  “ 

1160  DRAW“b»210, 150 ;. rial 

1162  DRAM" ba245 , ISO ; xan$ ;  “ 

1164  DRAM"bil5,130;xnp*;' 

1166  DRAH"b*45,130;x*eJ;‘ 

1163  DRAW'bi75,130;x*rJ;" 

1170  DRAW"b*lI0,130;XBS$;" 

1172  DRAM"biil40 , 180 ;  xaoS ;  * 

1174  DRAW‘bil70,lG0;x5in$;" 

1176  DRAU"ba200,130;x«nt;‘ 

1173  DRAU“b*2j0,130;x«eS;‘ 

1130  DRAW"b«265,180;XiU;" 

1102  GOSUB  50 

1P9  ’ . secretary  of  army  research  arid  study  fellowship  — . 

1200  CLEAR 

1201  MAI-"U5E10R3F10D5L3U5L1795L3BU?SR6E5RlF5Lir 

1202  HFJ:,!J15R22D3L15D3RIID3LUD6L3" 

1203  NRt-'U15R20F2D5G2L6F6L3H6L7D6L3EU88R3R13E2UlH2L13D5" 

1204  M35 - ' R16E2U3H2L 1 6H2U2E2R I4F2R2U2H2LL8G2D5F3R1 5D3L16H2L2D2F2R3 " 

1205  CLS: COLOR  3,1 

1206  DRAW"C3ba75,30(9Sj;":PAINTI7’,2,4) ,2.3 
120^  2RAW"C3b®!00,30x*aI; ":PAINT(101 ,291,2,3 
12C3  DRAW*C3b«130,30:<iirI:‘ : PAINT!  131,29) ,2,3 

1209  0RAW"C3b#l65,30x»sI; “.-PAINT (16b, 29) ,2,3 

1210  DRAW*33b#195,30:(«f J;":PAINT(  19fc,29)  ,2,3 


1211  DIM  ARSH35) ,ARA*!35) ,ARRf (35! , AfiF# ( 35 '•  .ARE If (35! 

1212  GET  i65,5)-(95,35!,ARS» 

1213  GET!9‘,5)-!125, 35), ARAB 
1211  GET !  125 , 5 )  - !  155 , 35 ! ,  AP.R4 
1215  SET !190,5)-(220, 35) ,ARFi 
121-.  5ET(  125,55)  155,35) ,  ARB  * 

1217  GOSUB  50: IF  BXUPS: “B"  OR  BKUPt: V  THEN  1000 


y-.\  v-vvw; 


1213  XX2-5 : FOR  UZ- 65  TO  40  STEP  -1 :PUT(ZZZ,XXZ> ,ARSI,PS£T:NEXT 
121*)  PUT (**5,5)  ,ARB» ,PSET 

1220  PUT (40,53) , ARAB, PSET 
122!  FOR  Z:1  TO  500:NEXT 

1222  PUT ( 125,5) , ARB* , PSET 

1223  PUT (40, 35), ARM, PSET 

1224  FOR  2:1  TO  500:NEXT 

1225  PUT ( 155,5) , ARBI, PSET 
1220  PUTUO,  110), ARSI, PSET 
122?  FOR  Z-l  TO  500:NEXT 
1223  PUT ( 190 , S) , ARBI, PSET 

1229  PUT140, 140), ARF», PSET 

1230  FOR  2:1  TO  500:NEXT 

1231  LOCATE  4,U:PRINT  ‘E  C  R  E  T  A  R  Y‘ 

1232  LOCATE  6,7:PRINT  ‘OF  T  H  V 

1233  LOCATE  10, 11:PRINT  ‘R  H  Y* 

1234  LOCATE  14,11:PRINT  *E  S  E  A  S  C  H  AN0‘ 

1235  LOCATE  17, 11  .-PRINT  ‘T  U  0  V 

1236  LOCATE  21,11:PRINT  ‘E  L  L  0  U  S  H  I  P‘ 

1237  GOSUB  50 
1233  COLOR  7,0 

1239  LOCATE  1,1:PRINT  ‘ 

1240  PRINT  ‘ 

1241  PRINT  ‘ 

1242  PRINT  ‘CONDUCTED  FROM  THE' 

1243  PRINT  “ 

1244  PRINT  ‘ 

1245  PRINT  ‘  CIVPERCEN 

1246  PRINT  ‘ 

1247  PRINT  ‘ 

1243  PRINT  ‘  FIELD  OFFICE' 

124')  PRINT  ' 

1250  PRINT  ‘ 

1251  PRINT  * 

1252  PRINT  ‘  IN 

1253  PRINT  " 

1254  LOCATE  17,11: PRINT  ‘A  N 

1255  LOCATE  2I,11:PRINT  “R  A  N  C  I  S  C  0 
125o  GOSUB  SO 

1257  CLS 

1 253  DRAW 'C35s?5 .JOxasS : ‘ : PAINT! 77,29) , 2, 3 
125')  DRAH‘C3bllOO,30xia$;‘:PAINTU01,29),2,3 
12*,0  DRAH*C3b»130,30xir J; ‘ : PAINT (131,29)  ,2,3 

1261  DRAW‘C3bll65,30xiSt;‘:PAINT(l66,2')),2,3 

1262  DRAU‘C3b*l l’5,30»if$;'': PAINT !  1%, 29) ,2, j 

1263  GOSUB  50: IF  BXUP$:*8*  OR  8KUFJ:‘b‘  THEN  1200 

1214  LOCATE  8,8 : PRINT  ‘6  MONTH  PROJECT GOSUB  50 
1265  LOCATE  10,3: PRINT  ‘FUNDED  UNDER  L  T  T‘:GOSUB  50 
12v,  LOCATE  I2,10:PRINT  ‘SALARY* : GOSUB  50 

12o'  LOCATE  14,10: PRINT  ‘EQUI'MEN-  ’ : GOSUB  50 
1263  LOCATE  lo,i2:PRINT  ‘MICROCOMPUTER " : GOSUB  50 
12-,')  LOCATE  13,12: PRINT  'SOFTUARE' :G"SUE  50 
12'0  LOCATE  20,0: PRINT  “ABOUT  *44.000  TOTAL  : GOSUB  50 
12’1  for  ::M2  to  20 : LOCATE  II:.S:PRINT  ‘ 


TE  12, OmRINT  “E’ AMINE  E--3YS  TECHNOLOGY” :G9SUB  50 


11  11  • '  l*  K 


F7 

,'-V. 

ft? 


«  -  "'1.  ■  '  W".  «  .  ■  1  Ul  l  1  ■■■ 


12?3  LOCATE  14,3:PRIHT  TOENTIFY  EXPERTS’ :G0SUB  SO 

1274  LOCATE  16,0: PRINT  “CONDUCT  COST  ANALYSIS* .GOSUB  50 

1275  LOCATE  13,8:PRINT  “DEVELOP  DEMO  NODULE  OF  EX -SYS * : GOSUB  50 

1276  LOCATE  20,8: PRINT  "DEFINE  DEMONSTRATION  PROJECT *: GOSUB  50 

1500  ’ . what-xsy  — . — 

1301  CLEAR.-SCREEN  1 : WIDTH  4O:CLS:C0LOR  10,0 

1302  LINE  (0,0) - (319 , 19**) , ,B 

1303  LOCATE  4,12:PRINT  “WHAT  ARE 

1304  LOCATE  7,5:PRINT  ’EXPERT  SYSTEMS'7 
1335  GOSUB  50: IF  BKUP$-“B’  OR  BKUPJ-’b*  THEN  1200 

1306  LINE  ( 1 , 70)- (319, 70) 

1307  LOCATE  12.3.-PRINT  ’->  Coiputer  Pr ogr a«s* : GOSUB  50 

1309  LOCATE  12,8:PRINT  ’  * 

no15  LOCATE  14,10: PRINT  ’:>  Use  Large  Knowledge  Bases” : GOSUB  50 

1310  LOCATE  14,10: PRINT  ’  ’ 

1311  LOCATE  16,10: PRINT  ’:>  Rules  and  Inference' :G0SUB  50 

1312  LOCATE  16,10:PRINT  ’  ’ 

1313  LOCATE  13,10:PRINT  ’-->  Eiplay  Self  Knowledge ’: GOSUB  50 

1314  FOR  ZM4  TO  1S.-L0CATE  22,10,-PRINT  ’ 

1315  LOCATE  14,8: PRINT  ’:>  Siiulate  Experts’ :GOSUB  50 

1316  LOCATE  14,S:PRINT  ’  * 

1317  LOCATE  16,3:PRINT  *:>  Use  by  Non -Experts “.-GOSUB  50 
1313  LOCATE  16,3: PRINT  ’  * 

1310  LOCATE  18,3: PRINT  ’=>  Give  Expert  Solutions" :GOSUB  50 

1320  LOCATE  13,9: PRINT  ’  » 

1321  ’ —  growing  box  to  clear  screen . 

1522  GOSUB  50 

1323  XI : 144 :X2x 170:41 -00:12: 110 

1324  FOR  21:1  TO  20 

1325  LINE!X1,Y1)-(X2,Y2) ,  1 , BF 

132*  <l:Xl-3:X2:X2*3:Yl:Yl-5: Y2:Y2^5 
1327  NEXT 

1323  CLS: COLOR  6,1 

1327  LOCATE  5,2:PRINT  ’HISTORICAL  0  V  E  R  V  I  E  M*: GOSUB  50 

bxupj.-t  or  exupj--'b’  then  1300 

1330  LOCATE  10, 10:PRINT  “MID- I°60’s  AI  RESEARCH’: GOSUB  50 
1531  LOCATE  !2,iO:?RINT  “LAWS  OF  REASONING* : GOSUB  50 

1332  LOCATE  !4,iO:PR!NT  ’POSSESSED  KNOWLEDGE GOSUB  50 

1333  LOCATE  16,10:?RINT  “EXPERT  MODEL  NECESSARY’ : GOSUB  50 

1500  ’ . practical  applications . . 

1510  CLE4R: SCREEN  1 : WfOTH  40: CLS: COLOR  5,1 

1511  MAX:  “U5E10R3F 10D5L3U5L1 7D5L3BU7BR6E5R1F5L1 1 ’ 

1512  MCI : *H2U1  IE2R20F2D2L3U2L 16G2D7F2RI6U2R3D2G2L 19’ 

1513  MIS: ‘U2F.5U1 1L5U2R1 3D2L5DI 1R5D2L1 3“ 

I3U  MLS : 'U1SR3DI 3R1302L20” 

1515  HNj:‘'JI5R3FI3U13R3D15L5HllDllL3’ 

151:  !'’d--*H2UlIE2RlSF2DIIG2Li3BE2BRIH2U7E2RL2F2D7G2LI2‘ 

1517  MFS  -  ‘U!  5F.20F2D6G2L!  -.DSL  T-EU^BRSF,  1 3E2U2H2L 13D6" 

1513  NRt :  ’  u  1 5S20F 205G2LpF *L  3H*>L  7DeL  ~8US8  R3RI 3E2U IH2L13D5* 

15P  “SJ:'R16E2U3H2L!6H2U2E2R14F?R2-,2H2L13G2D5F3R15D3LI6H2l2D2F2R3“ 

1523  MT}: 1 U 1 2L3U3R22DJL 7D 1 2L 4 ' 

1521  ’ - - - - - - . 

1523  DRAW“EM25, 35: 

1  524  J.RAN'BM'a, 35:  “ 

152!-  DRAW'8K32,35:«1AJ;' 


F-6 


Jut. 


‘.-NEXT 


:  IF 


1526  DRAH“BM114,35; XHCt ; “ 

1527  DRAH“BH152,35;XHTt;’ 

1523  DRAH'BH174,35;XH!t;‘ 

152'1  DRAW‘BM197.35;.XMCt;  * 

1530  DRAM " BM224 , 35 ; XHAt ; " 

1531  DRAH"8N256,35;Xt1LJ ;  ‘ 

1532  DRAW'BH5,&0;XHAJ; ' 

1533  DRAN“6H33,60;XHPJ;‘ 

1534  DRAH“BM60, 60 ; XMPt ; ' 

1335  DRAM‘BM37,60;XML$;" 

1536  DRAH"BM114,60;XMI$; " 

1537  DRAH'BH134,60;XHCJ; " 

1533  DRAW"BH160,60;XMAJ; " 

1539  DRAW“Bt1190,/j0;XMT$;“ 

1540  DRAU*BK210,60;XMI$;“ 

1541  DRAW*BM232 , 60 ; XMOt ; " 

1542  DRAW “8N260, 60; XMNJ ; “ 

1543  DRAW"BH2',0,6Q;XMSJ;‘ 

1544  SOSUB  50 : IF  BKUP^V  OR  BKUPt=“b‘  THEN  CLEARiGOTO  1323 

1545  LOCATE  13,10:PRINT  ‘H  E  D  I  C  A  L*:G0SU8  50 

1546  LOCATE  15,12:PRINT  "MYCIN* :G0SU8  50 

1547  LOCATE  15,18:PRINT  infectious  disease" .-G0SU8  50 
1543  LOCATE  17,12:PRINT  "CASNET" : GOSUB  50 

1549  LOCATE  17,19:PRINT  *-  glaucoaa" :GOSUB  50 

1550  LOCATE  19,12:PRINT  "CADUCEUS" :GOSUB  50 

1551  LOCATE  19,21:PRINT  internal  aedicine" :G0SU8  50 

1552  FOR  ZZJ.-15  TO  19.-L0CATE  ZZ2,12:PRINT  ’ 

1553  LOCATE  16,10:PRINT  "G  E  0  L  0  G  I  C  A  L*: GOSUB  50 

1554  LOCATE  18, 12:PRINT  “ PROSPECTOR* : GOSUB  50 

1555  LOCATE  13,23:PRINT  *-  linerai  explor * :GOSUB  50 

1556  LOCATE  20,12:PRINT  "Dipaeter  Advisor ' :GOSUB  50 
1537  LOCATE  20, 29 .-PRINT  *-  oil  wells*:G0SU8  50 
1553  FOR  ZZX-18  TO  2O:L0CATE  ZZ2,12:PRINT  ’ 

1550  LOCATE  19, 10 : PRINT  ’0  T  H  E  R‘:G0SUB  50 

1560  LOCATE  21,12:PRINT  *R  l’.SOSUB  50 

1561  LOCATE  21 , 16: PRINT  configure  coiputers" :GOSUB  50 
1362  LOCATE  23, 12; PRINT  “DENDRAL'; GOSUB  50 

1563  LOCATE  23,20:PRINT  analyze  coapounds" :GOSUB  50 

1564  FOR  ZZt-21  TO  23: LOCATE  ZZ2, 12;PRINT  ' 

1565  GOSUB  50: IF  SXUPt-'8*  OR  6KUPJ:‘b*  THEN  CLEARiGOTO  15L0 
1570  RUN  'DEMGRAPH* 

2000  1  cpo  budget  showing  'T5Z  salary  52  other 

2001  CLEAR:P!:3. 141593 

2002  CL3: SCREEN  1 ; WIDTH  40:C0L0R  0,1 

2003  LOCATE  2,15:PRINT  "CPO  BUDGET* 

2004  CIRCLE!  150, "01,100,,,, 10/13 

2005  IF  INRE'r J- "*  THEN  2005 

2006  CIRCLE ( 150 , 120  MOO , ,  1  rPI , P I >2 , 10/ 13 

2007  LINE '  50 , 95) - < 50 . 125 ) 

2003  LINE (250,45 } -1 250, 125 1 
23(N  PAINT  1150,150: ,2,3 

2010  CIRCLE : 150, ^Qi .  100,0, -PI/ 1.2,  -PI, 10/ 13 

2011  CIRCLE! ISO, 90s. ICO. 3, -PI, *P!/1. 2, 10/10 

2012  CIRCLE ! 130 .33! . 100 , , -PI/ I . 2 , -r I , 10/ 13 


2014  LINE ( 130 ,33)  - ( 130 ,'?0) 

2015  PAINT ( 100,30) ,1,3 

2016  PAINT (40,95) ,  1 ,3 

2017  PAINT(12Q,00),3,3 

2013  LOCATE  11,23:PRINT  'SALARIES' 

201?  S0SU8  50 

2020  LOCATE  13, I:PRINT  'OTHER' 

2021  SOSUB  50 

2022  LOCATE  9,1:PRINT  'TOY' 

2023  LOCATE  8,1:PRINT  'SUPPLIES' 

2024  LOCATE  7,1,-PRINT  'EQUIP' 

2025  GOSUB  50 

2026  LOCATE  12 ,20: PRINT  ' 

2027  LOCATE  13,20:PRINT  ' 

2023  LOCATE  14,20:PRINT  ' 

2029  LOCATE  13,21:PRINT  ‘95  I 

2030  GOSUB  50: IF  8KUP$=’B'  OR  BKUP*-'b’  THEN  2000 

2100  ’ - cpo  costs  - growing  bar  chart,  3-d 

2105  CLEAR 

2111  M1J: *U2R5U1 0L2E3R2D1 3R5D2L1 3 ' 

2112  t15$='U2R13U7LlSU6R20D2L17D2RI7Dl  1L20' 

2113  M6$--'U15R20D2L17D4R17D9L20R3BU2R14U5L14D$' 

2114  NIt:'U2R5UllL5U2R13B2L5DllR5D2L13* 

2115  HLJ= “U15R3D13R13D2L20' 

2116  MH$ = 'U15R3F9R2E9R3015L3UI 1G9L2H9D1 1L3* 

2117  ND0L$:‘R16E2U3H2L16H2U2E2R14F2R2U2H2L13G2D5F3R15D3L16H2L2D2F2R3BH+3, 
15U3R3D3R5U3R3D3BD15D3L3U3L5D3L3U3' 

2113  SCREEN  liWIDTH  40:CLS 

211?  LOCATE  1,12:PRINT  'CPO  COST  TO  ARHY":LOCATE  2,14:PRINT  ’!!  Millions) 

2120  M-0:F0R  22:22  TO  4  STEP  -3.-L0CATE  ZI,1:FRINT  H:M:M+20: NEXT: GOSUB  50 

2121  FOR  AM  70  TO  33  STEP-1:’  set  row  location  and  height  of  1st  bar 

2122  PSETUO.AZ) : ’  set  coluen  location  of  1st  bar 

2123  DRAM  'clR50c0elclE15':’  draw  angle 

2124  NEXT:’  repeat  til  height  is  achieved 

2125  DRAW“clL50GI5* rLOCATE  4,3:PRINT  "117.3':’  fill  in  top  of  bar 

2126  LOCATE  23,7:PRINT  'PROF* .-GOSUB  50 

2127  VIEH1 160, 80) '(300, 1501,2 

2128  GOSUB  50 

21215  LOCATE  12,24: PRINT  “GS-201  1062':G0SU8  50 

2130  LOCATE  13,24:PRINT  'GS-221  =  3I9':G0SU8  50 

2131  LOCATE  14,24:FRIHT  'GS-212  :  ?67':G0SUB  50 

2132  LOCATE  15,24:PRINT  'GS-230  =  320“:G03UB  50 

2133  LOCATE  16,24:PRINT  'G3-235  =  309':G0SUB  50 

2134  LOCATE  1 7 , 24 : PRINT  'GS-233  :  170':G0SUB  50 

2135  LOCATE  13,24:PRINT  '  Total  3647' :GOSUB  50 

2136  CLS 

2137  VIDJ 

2133  ’---bar  2— 

2139  r;,R  a:;170  n  l 30  STEP-1 

2140  »setuio,a;>,i 

2141  DRAW  “Cl R50c Oel c I E15" 

Ill?  HE <1 

2143  DRAM T1L50G15" :l)CATE  If , I7:PRINT  '35.3' 

2144  LOCATE  22,15: PRINT  'ADMIN' :GO;U:  SO 

2145  '—bar  3— 


F-8 


2146  FOR  A!- 170  10  165  STEP-1 

2147  PSET ( 180,AZ) ,2 

2143  ORAM  'ClRSOcOelclElS' 

2149  NEXT 

2150  DRAWC1L50G1 5":  LOCATE  20,26:PRINT  "4.4“ 

2151  LOCATE  23,24:PR[NT  'CLERIC* :G0SU8  50 

2152  ’—bar  4— 

2153  FOR  AX-170  TO  160  STEP-1 

2154  PSET ( 250 , AX) , 3 

2155  DRAW  “C2R50c0elc2E15" 

2156  NEXT 

2157  DRAN"C2L50G1S" :LOCAT£  20,35:PRINT  "3.3" 

2153  LOCATE  23,33:PRINT  "OTHER" :G0SU8  50 
215’ 

DRAW'c3bil30,60xidoll  ;b#160,60xal$  ;bil80,60xa6$;b»210,60xB5i;b*250,60xjnJ;biii200, 
h0x«it;b*297,60x#l$;" 

2160 

PAINT (131 ,59) ,2, 3: PAINT (161 ,59) ,2, 3:  PAINT (131, 59) ,2 ,3:  PAINT (211,59) ,2, 3: PAINT (25 
1,59)  ,2, 3: PAINT (231, 59) , 2 , 3 : PAINT (293 , 59) , 2 ,3 :COLOR  14,0 
2161  G0SU8  50: IF  8KUPX=‘8‘  OR  8)CUP$-'b"  THEN  2100 

2200  ’ . (paes)  logo . 

2201  CLEAR: SCREEN  1 ; WIDTH  80:CLS 

2202  DATA  '8M220 .30R15BR14R15' , *BH70 ,62R18* , "BM310, 53R23' , 'BH134 ,42R16BR50R16* 
2204  DATA  “BH70.61R13" , "8N200,32R66" , *8H193,33R73‘ , *8N184,43R22BR33R22" 

2206  DATA  ‘BM213, 58R158R20R20" , "BM438, 34R7I ‘ , "BM2I0, 3IR338R8R15 " , "BH308, 52R22 ' 

2203  DATA  "BH450 .31R55" , "BH511 .55L70" , "BN441 ,33R69" , "BH70.63RI3" 

2210  DATA  "BN184 , 38R30BR22R30" , ‘BH436 , 35R71 * , "BH314 , 54R25" , "BH310 , 34R33’ 

2212  DATA 

“BM183, 34R78" , '3M70, 65R18" , "BH185,35R81 " , "BM70 ,64R18“,"BM184,43R15BR52R15" 

2214  DATA  "BM305 , 51R21 " , 'BM134 , 36R32' , "8M320 , 55R71 * , 'BM134 , 39R26BR30R26* 

2216  DATA  "BH501 , 59Lb4“ , "BH184 .40R22BR33R22" , "BN435,38R30" , "BM71 ,35R73‘ 

2213  DATA  •BN73,34R75","BH70,37R23BR27R26’,"8H505,58L69","BN299,44R95" 

2220  DATA  "3M184 , 44R15BR52R15* , "BH302, 50R20" , “BM135, 53R31 ' , 'BN134 , 49R268R30R26' 

2222  DATA  "8M299,42R30BR46R23‘ , "BH315.33R7S" , "BM299, 43R97' , "BH70 , 51R13BR4R 40" 
2224  DATA  "BH70,43R74\"8!i299,45R93V6N184,52R82\"BI177,b9R4YBH505,S0L25" 

2226  DATA  4BI1435,36R70‘,"BH210,57R30BR12R13","BH43S,37R65*,'BH134,45R158R52R15” 

2223  DATA  'BH184 .47R18BR46R 18' , "8M510,56L71 " , 'BH438, 40R25" , "sH2’r) , 47R35' 

2230  DATA  "8H134,41R13BR46R13",‘BH70,49R70",*BM496,60L5S" 

2232  DATA  "8N184,50R308R22R30","8H322,32R62","BM184,51R34BR14R34","SM70,S0R66" 
2234  DATA  "B1151 1 , 54Lo5" , "8M70 ,52R138R12R23' , 'BH300 . 4'4R20‘ , '8H70,53R i3BR19Rl3" 
2236  DATA  "BH1S3.54R7S* , “8M452 ,44R25* , 'BH193 ,55R74‘ , "3H200, 56R50BR2RIt" 

2233  DATA  *8H294,4f>R39  YBH436,39R27"  ,"BM300,37R46BR12R41  YBN503,51L25" 

2240  DATA  “EM70,bt>R13“,"BH445, 42R25" , “8M184 , 46R 1 bBR50R16“ , “5M70 , 36R358R13R33' 

2242  DATA  “BM511 ,5JL30* , “BM70, 4CR19BR44R13' , "8M70, 47R/7" , 'BM134 ,37RJ4BR14R34 ' 
2244  DATA  " 9H345 , 30R1 5" , “BM70,57R18" , “BM330,31R45“ , "BH350,59R37” , "SN508,57L71" 
2246  DATA  'BM441 , 41R25"  r "BM71 ,67R16" ,  "BH70, 45R35BR13R33" , 'BH445 ,32Rt>4‘ 

2243  DATA  'BM306, 35R90' ,  “BH2?<»,43R20" , "EH2'4'4 , 33R41BE25R35' , ’BH45S, 3CR4I " 

2250  DATA  ’BH70,44R23BR27R26","BM73,63R12‘,"B!170,42R20BR42Rr?","B!170,46R74" 

2252  DATA  "BM299 , 39R37BR33S31' , “BH5 10, 52L27" , 'BHA^t , 43R24” , ‘BM501 , 49L25' 

2254  DATA  "8H70,33R22BR33R21",*BH483,61Ul","B«70,34R20BR42Ri4","BH45b,45R26" 
2256  DATA  "3H70, 5cRlS  Y3H303,36R94  YB,‘ 1490. 471.25", "8fPC,59i;i3\ '8H35,31R5r 
2252  DATA  *8(132’ . S0R66" .  "6H332 .S7R60' , "BN70, 54R10' , *SN340 .253R3G' . 'BN •  0 , 55R1S' 
2260  DATA  '?'4100,24R15",“?f170,5':R13’,“£N''l,30R35’,"Bf!7-':l.40R35BR33R28" 

2262  DATA  "BWS . 43L24" , "6H80 . 32R63" , "BN76 , 33R7C* , "BH244 . 41R32BR43R25" 

22’ 4  Data  “BH70,41R19BR44R10" ,  'BK70, t-ORlO'" , “8M70 , 43R22BR38R21 " , "E!143o,46L26" 


2266  FOR  ZZ=1  TO  132 
2263  READ  PAESJ 
2270  DRAW  *XPA£S$;’ 

2272  NEXT 

2274  PI=3. 141493 
2276  FOR  ZS- 100  TO  108 

2273  CTRCLE( 21 ,50) ,30, ,PI/l.5,1.33*PI 
2280  NEXT 

2232  FOR  ZZ--430  TO  472  STEP-1 
2284  CIRCLEfZZ, 50) ,80, ,1.65*PI, PI/2.9 
2236  NEXT 

2238  DIM  ARI(545):GET(15, 19) - (565, 81 ) , ARI 
22^0  SCREEN  1:HIDTH  40:CLS:C0L0R  5,1 
2292  PUT (7,19) , ARI 

2294  LOCATE  15,7:PRINT  "A  PERSONNEL  ADMINISTRATION* 

22%  LOCATE  17,13:PRINT  "EXPERT  SYSTEM* 

22^3  LOCATE  19, 7: PRINT  ‘FOR  THE  DEPARTMENT  OF  ARHY* 

2299  GOSUB  50 

2300  ’ —  paes,  full  blown  systes  - 

2302  CLEAR: SCREEN  1 :HIDTH  40:CLS:C0L0R  6,1 

2304  LOCATE  2,9:PRINT  "TOTAL  (paes)  OPERATION" :GOSUB  50:IF  BKUP|--"8*  OR 
BKUP$:*b"  THEN  2200 

2306  LOCATE  7,5:PRINT  "Cover  ALL  Functions* 

2303  FOR  Z-l  TO  5:L0CATE  7,2:PRINT  *  *:F0R  22=1  TO  200: NEXT: LOCATE  7,2:PRINT 
“ - ) " : FOR  ZZ--1  TO  300: NEXT ; NEXT : GOSUB  50:L0CATE  7,2:PRINT  *  *“ 

2310  LOCATE  10,5:PRINT  "Directly  Operated  by  Supv/Eipl" 

2312  FOR  Z-l  TO  5:L0CATE  10,2:PRINT  "  *:FOR  22=1  TO  200: NEXT : LOCATE  10,2:PRINT 
*:>":FOR  ZZ:1  TO  300: NEXT: NEXT: GOSUB  50: LOCATE  10, 2: PRINT  "  »* 

2314  LOCATE  13,5:PRINT  "Decisions  Made  by  (paes!* 

2316  FOR  Z:1  TO  5:L0CATE  13,2:PRINT  "  *:FOR  U'-l  TO  20O:NEXT:L0CATE  13,2:PRINT 
*:)*:FOR  ZZ=1  TO  300: NEXT: NEXT: GOSUB  50:L0CATE  13,2:PRINT  *  4" 

2313  LOCATE  16,5:PRINT  "Appropriate  Actions  Taken  by  (paes)" 

2320  FOR  Z:1  TO  5:L0CATE  16,2:PRINT  *  ":FOR  2Z=1  TO  200:NEXT:L0CATE  U,2:PRINT 
":>*:F0fi  ZZU  TO  300:NEXT:NEXT:G0SUB  50:L0CATE  16,2:PRINT  *  »" 

2322  CLS: COLOR  0 

2324  LOCATE  1,16:PRINT  “EXAMPLE" :G0SU6  50:IF  BXUPi=’B"  OR  BXUPt-"b*  THEN  2300 
2326  LIME { 0 . 10) - ( 60 , 35 ) , 2 , BF 

2323  LOCATE  4,9;pRlNT  "Superv  gets  new  manpower  auth":GOSUB  50 
2330  LOCATE  4,9:PRINT  " 

2332  LOCATE  3,3:PRINT  "New'.-LOCATE  4,3:PRINT  ’Job* 

2334  DRAH"clbl30,36dl3rlul3l2dl3h4r3g4" 

2336  LINE ! 0 , 50) - ( 60 , 75) ,2 ,BF 

233?  LOCATE  9,9:PRINT  "Use  (paes)  froi  ofc  terainal* :GOSUB  50 
23^0  LOCATE  V:PRINT  * 

2342  LOCATE  3,2:PRINT  "Access’: LOCATE  °,2:PRIMT  *(paes)‘ 

2344  DRAH"clb»30,76dl3rlul3l2dl3h4r8g4" 

2346  LINE!0,'?0)-(60,115) , 2 , BF 

234?  LOCATE  14,'>: FRINT  "(paes)  proepts  supv  for  info“:GOSUB  50 

2350  LOCATE  14,9:?RINT  * 

2352  I'-CATF  !3,2:FR'NT  *(paes)*:LOCATE  14,2:FRINT  "Proept" 

2351  ORAW'c  Ifc*30, 116*11  Jr  Iul3l2dl3h4r3g4* 

235*  L:?iE;0.l30Hr,0,155),2,BF 


'?  l'  CAFE  [’>.6 
LOCATE  IV 
-,2  LOCATE  19.3 


PRINT  "Superv  responds  with  incut ":GOSUB  50 
:RIMT  * 

PRINT  "Sup-.'; LOCATE  1'5,2:?RINT  "Reply?* 


855 


DRAH'c lb*61 , 142r 15u40r ld40r lu40l 17e4d8h4 “ 

LOCATE  16,11: PRINT  ‘Interaction' :G0SU8  50 
LOCATE  16,11:PRINT  ' 

DRAH'c0b«61,l42rl5u40rld40rlu40ll7e4d8h4' 

DRAH‘clb«30,15fcdl3rlul3l2dl3h4rSg4‘ 

LINE (0, 170 )-(6Q,195),2,BF 

LOCATE  24 , 9 : PRINT  "(paes)  decides  classification" ; :GOSUB  50 
LOCATE  24,9: PRINT  *  *; 

LOCATE  23,3: PRINT  ‘Job‘:LOCATE  24,2:PRINT  'Alloc'; 

DRAM'c lb*61 , 133r 58h4d3e4* 

LINE (120, 170) -(180, 195) ,2, BF 

LOCATE  20 , 15:PRINT  'Candidate  evaluation" 

LOCATE  21,16: PRINT  'established  by  (paes)':GOSUB  50 
LOCATE  20 , IS : PRINT  ' 

LOCATE  21,16,-PRINT  ' 

LOCATE  23, 1S:PRINT  'C-£*:LOCATE  24,18:PRINT  'Docs'; 

DRAH'c Ibil31,133r68h4d8e4‘ 

LINE(250,170)-(310,195) ,2,BF 

LOCATE  18,17: PRINT  "Perforiance  ianageeent' 

LOCATE  19.13.-PRINT  'criteria  established'.-GOSUB  50 
LOCATE  18,17: PRINT  ' 

LOCATE  19, 13: PRINT  ' 

LOCATE  23,34:PRINT  'P-N'rLOCATE  24,34:PRINT  'Docs'; 

DRAH'c Ib«230,169ul3rldl312ul3g4r3h4‘ 

LIN£(250,130)-(310,155) ,2,BF 

LOCATE  18,12: PRINT  ‘ (paes)  searches  for' 

LOCATE  19,13:PRINT  'best  qualif  cands':GOSUB  50 
LOCATE  13,12: PRINT  ' 

LOCATE  19,13: PRINT  * 

LOCATE  13,34: PRINT  *Cand*:L0CATE  19,33:PRINT  'Search* 

DRAH'c Ib*230,129ul3rldl3l2ul3g4r8h4' 

LINE (250, 90) -(310, 115) ,2,8F 
LOCATE  13,15: FRINT  "BQ  list  provided" 

LOCATE  1 A ,  16 : PRINT  "to  supervisor' : GOSUB  50 
LOCATE  13,15:PSINT  " 

LOCATE  14, 16:PRINT  * 

LOCATE  13, 35:PRINT  'BO':LOCATE  14,34:FRINT  'List' 
DRAH'clb#230,39ijl3rldl3i2ul3g4r3h4' 

LINE (250,50) -(310, 75) ,  2 ,  BF 

LOCATE  9, 12: PRINT  'Supervisor  selects". -GOSUB  50 

LOCATE  9,12: PRINT  ' 

-  LOCATE  3, 34: PRINT  *Supv':LOCAT£  9,33:PRINT  'Select* 

:  DRAH'c Ib#230,49ul3rldl312ul3g4r3h4* 
i  LINE(250,10)-(3i0,35),2,BF 
1  LOCATE  3 , 10 : PRINT  “Notifications  *ade  and" 

1  LOCATE  4 , 10 : PR INT  'actions  done  by  (paes) ' :G0SU6  50 
i  LOCATE  3, 10: PRINT  " 

I  LOCATE  4,10: PR INT  ' 

I  LOCATE  3,33:?RINT  "Ac ti on " .-LOCATE  J,33:PRINT  'Taken” 

'  GOSUB  50 

I  ’ . hypotheses  . 

1  CLEAR :SCREEN  i:HIDTH  40:CLS: COLOR  1-1,0 
LOCATE  2,10:  PRINT  “HYPOTHESE  S':GO«VS  50:  IF  STOPS:  V  OR  ETUFJ: 


L OCATE  7,5: PRINT  “1.  E/S  Technology  can  be  applied' 


•  _  *• 

>v- 

>> 


2503  LOCATE  3.3.-PRINT  "to  Personnel  Ad*inistration‘ :G0SUB  50 
2510  DIH  AR# ( 671 ) 

2512  GET{ 75, 30) -(235, 160), API 

2514  VIEW  (75,30)-(235, 160) , 1 : ’viewport  for  info  about  II 
2516  LOCATE  5,17:PRINT  ' 

2513  LOCATE  6,17:PRINT  *  BASIS  " 

2520  LOCATE  7, SPRINT  "  ":G0SU8  50 

2522  LOCATE  M3:PRINT  "P-A  is  Knowledge" 

2524  LOCATE  10,13:PRINT  "  intensive  * : GOSUB  50 

2526  LOCATE  12,13:PRINT  "  -E/S  directly  * 

2523  LOCATE  13,13:PRINT  ‘  apply  ":GOSUB  50 

2530  LOCATE  15,13:PRINT  "P-A  is  'Rules’  * 

2532  LOCATE  16,13:PRINT  "  oriented  ":GOSUB  50 

2534  LOCATE  18,13:PRIHT  "  -E/S  well  suited" : GOSUB  50 
2536  CLS:VIEH: ’close  viewport. 

2533  PUT (75,30) , ARir 60SUB  50: ’restore  screen 

2540  LOCATE  10,5:PRINT  "2.  If  applied,  dollars  can  be" 

2542  LOCATE  11,8:PRINT  ‘saved  by  reducing  personnel " : GOSUB  50 
2544  GET(75,30)-!235, 160) ,ARI 
2546  VIEW  (75,30)- (235, 160) , 1 
2548  LOCATE  5, 17: PRINT  ‘ 

2550  LOCATE  6, 17: PRINT  *  BASIS  * 

2552  LOCATE  7, 17:PRINT  *  ": GOSUB  50 

2554  LOCATE  M2: PRINT  "Supervisors  use  * 

2556  LOCATE  10,12:PRINT  *  E/S  directly  ":GOSUB  50 

2553  LOCATE  12,12:PRINT  "E/S  autoiatically" 

2560  LOCATE  13,12:PRINT  "  accoiplish  work  ":G0SUB  50 
2562  LOCATE  15,12:PRINT  “Reduced  nuiber  " 

-2564  LOCATE  16,12:PRINT  "  of  personnel ists“ :GOSUB  50 
2566‘C1.S:  VIEW(75,30)-(235, 160) ,  1 


2563 

LOCATE 

5,15: PRINT  * 

■ 

2570 

LOCATE 

6,15: PRINT  " 

HOW  HUCH’  " 

2572 

LOCATE 

7,15: PRINT  " 

"  .-GOSUB  50 

2574 

LOCATE 

^ , 13 : PRINT  " 

Currently  " 

2576 

LOCATE 

10,13:PRINT 

* - - - * 

L  J  •  J 

LOCATE 

1 1 , 13: PRINT 

"  6200  People  " 

GOSUB 

50 

2530 

LOCATE 

12, 13: PRINT 

2532 

LOCATE 

13, 13 :  PRINT 

“  1157,000,000  " 

GOSUB 

50 

2534 

LOCATE 

15 , 13: PRINT 

’  Possibly 

2586 

LOCATE 

16 , 13: PRINT 

2533 

LOCATE 

17,13: PRINT 

"  4500  People  " 

GOSUB 

50 

25’0 

LOCATE 

13,13: PRINT 

2592 

LOCATE 

19,13: PRINT 

"  1109,000,000  * 

GOSUB 

50 

25,i  CLS: VIEW 

25°b  PUT  (75 ,30) ,AR»: GOSUB  50 

25-3  LOCATE  I3,5:PRINT  "3.  If  iapienented,  responsibility" 

2600  LOCATE  U,3:PRINT  "for  PA  will  be  fixed  with  supvs": GOSUB  50 
2-,02  GET!75,30)-!235,lt.0),ARI 
2604  VIEU(75,30)-(235,lr>G),l 
2606  LOCATE  5.17.PRINT  * 

2600  LOCATE  6,17:FRINT  ‘  BASIS 

2610  LOCATE  7, 1 7: PRINT  *  *:50SUB  50 

2-12  LOCATE  3, 12: PRINT  "Supervisor  input  * 

2614  LOCATE  I0,I2:FRINT  ’  controls  E/S  ": GOSUB  50 

2b 16  LOCATE  12.12:PRINT  "Elaborate  safe-  * 


F-12 


ilV'% 


/  s 


2615  LOCATE  13.12.-PRINT  *  guards  are  not  * 

2620  LOCATE  14,12:PRINT  ’  efficient  ’:G0SUB  50 

2622  LOCATE  lo,12:PRINT  'Cost  savings 

2624  LOCATE  17,12:PRINT  '  are  essential  ’:G0SUB  50 
2626  CLS : VIEW 

2623  PUT (75,30) ,ARS:GOSUB  50 

2630  LOCATE  16,5:PRINT  ‘4.  Iiproved  consistency  of  PA  Mill" 

2632  LOCATE  17,3:PRINT  ’be  realized’ :GOSUB  50 
2634  GET(75,30)-(235,160),AR» 

2636  VIE»(75,30H235,160),1 
2638  LOCATE  5,17:PRINT  ’ 

2640  LOCATE  6,17:PRINT  ’  BASIS  * 

2642  LOCATE  7,17:PRINT  ’  ’:G0SU8  50 

2644  LOCATE  'M2:PRIHT  ’Syste*  operates  ’ 

2646  LOCATE  10,12:PRINT  *  sale  everywhere  ":GOSUB  50 
2648  CLS : VIEW 

2650  PUT (75,30), ARS : G0SU8  50 

2652  LOCATE  19,5:PRINT  ’5.  Tiieliness  of  service  will  be’ 

2654  LOCATE  20,3:PRINT  ’orders  of  lagnitude  better ’ :GOSUB  50 
2656  GET (75, 30) -(235, 160) , ARS 

2653  VIEW(75,30)-(235,160) ,1 

2660  LOCATE  5,17:PRINT  ’ 

2662  LOCATE  6,17:PRINT  *  BASIS  ’ 

2ft64  LOCATE  7,17:PRINT  ’  ‘.-GOSUB  50 

266s  LOCATE  12: PRINT  ’Cciputers  faster’ 

2663  LOCATE  10,12:PRINT  *  than  people  ’:GOSUB  50 
2670  LOCATE  12,12:PRINT  ’Systei  available’ 

2672  LOCATE  13,12:PRINT  ’  at  work  site  ’:G0SUB  50 
2674  CLS: VIEW 

2676  PUT (75,30 ),ARI: GOSUB  50 

2700  ’ . expert  systei  deio  lodule- . 

2702  CLEAR: SCREEN  1 :MIDTH  40:CLS:C0L0R  14,0 

2704  LOCATE  3,13:PRINT  ’EXPERT  SYSTEM’: LOCATE  4,10:?RINT  ’DEMONSTRATION 
MODULE’ :G0SU8  50: IF  BKUPt=’B’  OR  BKUPJ='b’  THEN  2500 
2706  LOCATE  7,11: PRINT  ’JOB  CLASSIFICATION’ 

2703  FOR  DM  TO  4:F0R  2ZZ=1  TO  3:CIRCLE  (60.501 , ZZZ.  1  :NEXT:FOR  ZZM  TO 
8:CIRCLE  (60,50) . ZZZ , 2 : NEXT : NEXT 

2 710  LOCATE  7,7:PRIMT  ’=>’:G0SU8  50 

2712  LOCATE  10,11:PRINT  “MIMICS  OPM  SYSTEM’ 

2714  FOR  OM  TO  4:F0R  221=1  TO  3:CIRCLE  (60,74),22X,l:NEXT:F<tR  ZZM  TO 
3 : CIRCLE  ! 60 , 74 ) , 111 , 2 : NEXT : NEXT 
2716  LOCATE  10,7:PRINT  *=)* :GOSUB  50 

2713  LOCATE  13,11:PRIHT  ’EXPERT  EASE  TOOL  USED’ 

2’20  FOR  g;=l  TO  4:F0R  221=1  TO  3:CIRCLE  (60.48) , Z21, 1  :NEXT:rOR  ZZM  TO 

?: CIRCLE  (60,93! ,221, 2: NEXT: NEXT 

2T22  LOCATE  13,7:PRINT  *=>’:G05UB  50 

2724  LOCATE  !t.,il:FRINT  ’GS  TOO  COMPLEX  FOR  TOOL’ 

2726  FOR  M=1  TO  4: FOR  221=1  TO  3:CIRCLE  (bO,123!,22S,l:NE:<T:FOR  22M  To 

?: CIRCLE  (60 , 123) , 22J ,2: NEXT : NEXT 

2723  LOCATE  i6,7:PRINT  ’  =  > ’ : GOSUB  50 

2730  LOCATE  IMhPRIHT  ’WAGE  GRADE  SYSTEM  USED’ 

2772  MR  OM  TO  4:F*»S  ZZM  To  -hCIitCLE  4eO, i47> 1  :‘lcVT:F'-*S  ZZM  ' 

SrCIMlE  i^.l4*!,rr:,2:NEXT:HEXT 

2734  LOCATE  i*. 7: PRINT  *:>*:GOSUB  50 

2’3  ’  - how  ;t  works - 


WWW 


2738  SCREEN  0 :CLEAR 
2740  SCREEN  hHIDTH  40:CLS 

2742  LOCATE  3,14:PRINT  'HOW  IT  WORKS' :GOSUB  SO: IF  BKUP$='B“  OR  8KUPS-V  THEN 
2700 

2744  ’ . draw  coaputer  . — 

2746  At='MH,-3M+l,-3M+2,-2Mt3,-l' 

2748  8$="M+3,+iM+2,+2M+l,+3M+I,+3“ 

2750  Ct--'M-l,t3M-2,t2H-3,HH-3,U* 

2752  D$:'N-3,-lH-3,-iH-2,-2H-l,-3* 

2754  DRAW  '  c  28M0 .  '^OUSOXA* ;  R30X3$ ;  D50XCS ;  L75XD$ ; 

2756  DRAW  ’ BH10 , 30U30XA5 ; R60X8  $ ; D30XCS ; L55XD5 ; “ 

2’58  ORAM  'BHO, I00R95D30L95U30* 

27t<0  ’ . keys  on  keyboard - 

27^2  FOR  QZ- 110  TO  125  STEP  5 
2764  FOR  ZMO  TO  30  STEP 

5;PSET(ZZ,0ZS :PSET(ZZ+L,QZ; :PSET(ZZ,iiZtI) :PSET(ZZ+l ,QZ+1) :NEXT 
2766  NEXT 

2763  FOR  ZZ=2S  TO  65  STEP  5: PSET( ZZ , 126) ,0: PSET ( ZZ+1 , 126) ,0:NEXT 
2770  DRAW  "C3BM25, 125R41D3L41U3' : ’ -  space  bar  - 

2772  FOR  Z- !  TO  500:NEXT 

2774  LOCATE  3, IMPRINT  'COMPUTES  PROMPTS  USER' 

2776  FOR  Z--1  TO  1500:NEXT 

2773  ’ . data  on  coaputer . 

2730  FOR  QI-55  TO  70  STEP  4 

2782  FOR  ZJ:20  TO  60  STEP  4.-PSET (ZZ, OZ)  :FOR  Z  =  i  TO  50: NEXT : NEXT : NEXT 

2733  GOSUB  50: IF  3KUP*:'8'  OR  8KUPJ:‘b'  THEN  2700 

2734  LOCATE  10,19,-PRINT  ‘USER  RESPONDS' .FOR  2=1  TO  1500:NEXT 

2736  ’ . blink  keyboard . 

2733  OMI5.-FOR  ZZ- 30  TO  70  STEP 

10:PSET(ZZ,QZ),0:PSET(Z:u,QZ),0:PSET(ZZ,QZH),0:PSET(ZZU,QZtl!,0:FOR  l-l  TO 
500: NEXT 

2790  PSET(ZZ,QZ) :PSETlZZ+l,QZ) :PSET(ZZ,QZ+1) : PSET v ZZ+ 1 , QZ+ 1 ) : NEXT 

27’52  ’ . data  on  keyboard  again . 

FOR  31-55  TO  70  STEP  4 

2'°6  FOR  ZZ-20  TO  60  STEF  4:FSET(Z:,QJ) ,0:NEXT:NEXT 

2'%  FOR  <:1  TO  3 

2300  FOR  31:55  TO  70  STEP  4 

7002  FOR  ZZ-20  TO  60  STEF  4:?SET(ZX,«) :FOR  Z  =  1  TO  50: NEXT  Z:NEXT  ZX:NE'<T 

2304  FOR  ‘5Z-55  TO  70  STEP  4 

2S06  FOR  73:20  TO  60  STEP  4: PSET(ZZ.QZ)  ,0:NEXT  it: NEXT  OX 

2303  LOCATE  12,i9:PRIMT  *ET  CETERA" 

2310  NEXT  X 

2812  LOCATE  14,19:?RIHT  'COMPUTER  CLASSIFIES” 

2814  LOCATE  15,19:PRINT  *JOB':FOR  Z=1  TO  1000:NEXT 
2816  LOCATE  7,4:FRINT  ‘M.V.O. ": LOCATE  5: PRINT  'WG-5':G0SUB  50 
2313  LOCATE  19,3:FRINT  ‘BASED  ON  OPH  STANDARDS' : GOSUB  50 
2820  LOCATE  21,8:PRINT  "USES  EXPERT  JUDGEMENT': GOSUB  50 
2322  ’ . aodule  structure  defined . 


2324  CLEAR: SCREEN  1: WIDTH  30:CLS 

2°2r.  LOCATE  2,t.:5RINT  ‘EXPERT  SYSTEM  DEMONSTRATION 
0  D  U  L  E‘ 

7029  GOSUB  50: IF  ?ryaj:V  OR  BXUPJ:'b"  THEN  2736 

2373  ’ - -jr  ok  square  in  siddle  of  screen . 

2372  •;X-2SO:vlX:iOO:v2:=320:Y2Z:I01:FOR  ZZX-I  TO  15:: T'-E C X 1Z, VIZ  = 
r<2J,  B:LINEl<i:-l,YtX-U-!X2M.Y2ZM)..B:LINE;Xi:,,'i:i- 


(X2ZtY2Z)l0,B:XlZ:XlZ~l:YlZ:YlZ~l:X?»:X2Z+l:Y2Z=y2Z+l:NEXT 
2834  LOCATE  L2,33:PRINT  "WAGE* 

2836  LOCATE  13,32:PRINT  'GRADE' 

2333  DIM  ARZ { 300 )  :GET  (235,35) -( 337 , 116) .ART 
2840  COSUB  50 

2842  1 . «ove  square  to  left  eoge . 

2344  FOR  221:235  TO  10  STEP  -2:PUT  (221,35) , ARZ , PSET : NEXT 
2846  LOCATE  14,4:PRINT  'start' 

2343  GOSUB  50 

2350  LINE  v200, 40 ) -  ( 300 , 70 ) ,  ,B 

2352  LOCATE  7,28:PRINT  'A  U  T  0':L0CATE  8,27,-PRINT  "R  E  P  A  I  R'rGOSUB  50 
2354  LINE (200, 130) -(300, 160) ,  ,8 

2856  LOCATE  13,28:PRINT  'F  0  0  D':LOCATE  19,28:PRINT  'P  R  E  P':GOSUB  50 
2358  DRAM  *BH1 12, 100R45U45R40H5F5G5' : GOSUB  50 
2060  DRAM  "BN157, 100D45R40H5F5G5" : GOSUB  50 

2362  GET  (260,0)-(600,20),ARZ: '-part  of  title 

2364  FOR  221=0  TO  15:PUT  (260,221) , ARZ, PSETrNEXT: ’-»ove  part  title  dour* 

286*  FOR  221:260  TO  0  STEP  -5:PUT  (221,15), ARZ, PSETrNEXT:' -non  love  it  left 

2363  GET  (0, 5H320. 30) , ARZ: '-whole  title 

2370  FOR  221:3  TO  0  STEP  -1:PUT  (0,221) , ARZ, PSETrNEXT: ’-non  love  it  up 
2372  LINE ( 400 , 1 3 ) - ( 500 , 33 ) ,  ,8 

2874  LOCATE  3,52:PRINT  'MOBIL  E* : LOCATE  4,53:PRINT  '£  0  S  V  C'rGOSUB  50 
2376  LINE(400, 33) -(500,53) , ,8 

2973  LOCATE  6.54.-PR1NT  'A  U  T  0" .-LOCATE  ?,55:PRINT  "1  E  C  H' .GOSUB  50 
2330  LINE ( 300,50) -(393,20) 

2332  DRAW  'H5D1F5G5D1E5' 

2384  FOR  221:1  TO  [OOOrNEXT 
2386  LINE(300,60)-(398,45) 

2333  ORAW  'H5D1F5G5D1E5': GOSUB  50 
2890  LINE(400, 180)-(500, 199),, B 

2392  LOCATE  24,S4:PRINT  'C  0  0  X* ;  rGOSUB  50 
2394  LINE (400, 155) -(500, 175) , ,8 

23q6  L0C7':  21 , 52 : PRINT  "F  D  S  V  C'-.GOSUB  50 

2393  LINE (300, 150) *(3q8, 138) : DRAW  'H5R1F5G5D1E5' 

?9QQ  FOR  221:1  TO  1000.-NEXT 

2902  LINE (300, 140) -(398, 165 ): DRAW  'H5R1F5G5D1E5' rGOSUB  50 
2904  LOCATE  9,64:PRINT  '0  p  e  r  a  t  0  r  s' 

2°06  LINE (400,70; -(500,  -2 ! , ,8 :LINE( 520 , 35 ; - ( t-20 , 105 ) , , B : LINE (520 , 1 10 J - 
( 620, 130),, B rGOSUB  50 

2q08  LOCATE  10,52:PRINT  "M  0  T  0  R'rLOCATE  11,52:PRIHT  'V  E  H  I  C  L'rFOR  221:1 
TO  1000: NEXT 

2910  LOCATE  12 , b8 r PRINT  *F  0  R  K * rLOCATE  13,63:PRINT  *L  I  F  T " : FOR  22M  TO 
1000 .NEXT 

?°12  LOCATE  1 5 , b3 : PRINT  'C  R  A  N  E' rGOSUB  50 

2°14  L INE U 53 . 30 ) * ( 393 , 30 ! : DRAW  'H5D1F5G5U1E5':F0R  221:1  TO  lOOOrNEXT 
2916  LINE  (1 58, 971- (519, 97):  DRAW  "H5D1F5G5U1E5 ' :  FOR  22M  T«  lOOOrNEXT 
2918  LINE i  158. 115) -(513, 115)  rDRAW  ’H5D1F5G5U1E5'  :FOR  221:1  TO  1 000  :NEU  rGOSUB  50 
2 "’20  LINE (400 ,125) -(500, 145),, 8: LOCATE  17,56:PSINT  *A  /  C” rLOCATE  13,55:FRINT  "N 
E  C  H' rGOSUB  50 

P22  l THE !  153 , 125 H  3*5 , 1 25 ) : DRAW  'H5D1F5G5U1E5' rGOSUB  50 

5000  ’ .  critical  issues  . - 

300?  Cl  EAR: SCREEN  IrWID^H  40:CLS:COLOR  1,1 
3C01  LOCATE  2, 12: PRINT  'CRITICAL  ISSUES' 

3G05  5  5-<c  30: IF  8XUP1-T'  -ifi  3FUPJ-  ‘5“  THEN  2322 
3G0t  LOCATE  ".SrPRINT  'MIMIC  CURRENT  SYSTEM?4 
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3003  LOCATE  8,8:PRINT  'SYSTEM  CHEATERS 
3010  LOCATE  *,8:PRINT  'IMPERSONAL  PERSONNEL' 

3012  LOCATE  10,3:PRINT  'COSTS  TO  IMPLEMENT' 

3014  LOCATE  11,8:PRINT  'IMPACT  ON  CAREERISTS* 

3016  LOCATE  12,8:PRINT  "HON  TO  DEVELOP" : GOSUB  50 

3013  ’ . store  each  issue . - 

3020  DIM  ARli (50): SET (55, 47) -(222 ,55)  ,AR1* 

3022  DIM  AR2I(5Q) :6ET (55 , 55) -(222,63) ,AR2i 
3024  DIM  AR3I(50):SET(55,63/-(222,71),AR3I 
3026  DIM  AR4t(50) :GET ( 55 , 71 ) - (222 , 79) ,AR4i 

3023  DIM  AR5I (50) :GET (55, 79) -(222, 87) ,AR5I 
3030  DIM  AR6I(50) :GET( 55 ,37) -(222,95) ,AR6* 

3032  ’ — store  issues  in  pouch - 

3034  DIM  ARK270) :GET(49,4?)-(221,95) ,AR» 

3036  FOR  ZZI=35  TO  110: PSET (227, 2Z2) :FOR  XXM  TO  25:NEXT:NEXT 

3033  DRAW 'BM227 , J5F5D6SGS* 

3040  PAINT  (229, 55), 2, 3 

3042  FOR  222-55  TO  225  STEP  7 
3044  PUT (222,47) ,AR«,PSET 
3046  GET(Z2I,47)-(223,95) ,ARI 

3043  NEXT 

3050  FOR  ZZ2=5  TO  15:L0CAT£  ZZ2,25:PRINT  “  *:NEXT 

3052  GOSUB  50 

3054  ’ — bring  the*  back  one  at  a  tiie . 

3056  FOR  ZZM90  TO  50  STEP-2: PUT(55, 2Z2) ,AR1J,PSET:NEXT 

3058  FOR  ZZM99  TO  59  STEP-2 : LINE ( 55 , ZZ2) -( 222 , 222) , 0 : NEXT : GOSUB  50 

3060  FOR  ZZM90  TO  75  STEP-2 : PUT (55 , 2Z2)  ,AR2I, PSET: NEXT 

3062  FOR  222=199  TO  34  STEP-2:LINE (55 , ZZ2) - (222 , 2Z2 ) ,0:NEXT : GOSUB  50 
3064  FOR  222=190  TO  100  STEP-2: PUT ( 55, 222) ,AR3« , PSET :NEXT 

3066  FOR  ZZX--199  TO  109  STEP-2:LINE(55,ZZ2)-(222, ZZ2)  ,0:NEXT:GOSUB  50 

3063  FOR  Z 22= 190  TO  125  STEP-2:PUT(55,ZZI),AR4I,PSET:NEXT 

3 IPO  FOR  ZZM99  TO  134  STEP-2:LINE(55,ZZ2)-(222,ZZ2)  ,0:NEXT:GOSUB  50 

3072  FOR  ZZM90  TO  150  STEP-2 : PUT ( 55 , ZZ2 ) , AR5f  ,PSET :NEXT 

3074  FOR  2ZM99  TO  159  STEP-2: LINE(55 , Z221- (222 , 222)  ,0:NEXT:G0SUB  50 

3076  FOR  ZZM90  TO  175  STEP-2: PUT (55, ZZ2 ) , AR6I ,P3ET NEXT 

3078  FOR  ZZJ--199  TO  104  STEP-2.-LINE ! 55, 2227 - (222, Z22) , O.-NEXT 

3080  GOSUB  50: IF  8XUPMV  OR  BKUFJ:'b'  THEN  3000 

3100  ’ - have  a  good  day  . 

3105  CLEAR :SCREEN  1:WIDIH  )0:CL3 

3107  LOCATE  3,5:PRINT  ”HAVE':LOCATE  5,7:PRINT  'A* 

3110  FOR  ZJ=95  T9  99 
3115  CIRCLE  (160, 100), ZJ 
3120  NEXT 

3130  FOR  ZM  TO  15 
3135  CIRCLE  1130, 75), ZJ 
3140  NEXT 

3145  FUR  ZM  TO  15 
3150  CIRCLF  (190,75),;: 

3155  NEXT 

3160  FOR  ll-%  TO  100 

3165  CIRCLE  UfeO.Ztl, 70,, 3.5.6 

31“Q  NEXT 

■;'5  1  ,iCATE  21, 34: PRIST  "GOOD" : LOCATE  23, 34: PRINT  'DAY' 

3200  PAINT  (it 0,100:. 2,3 

3903  G..SUB  50: IF  5‘UPMB'  OR  BiUPR’-'b”  THEN  3000 


4000  IF  8KUPJ:'Z’  OR  BKUP*='z'  THEN  RUN  ‘BUM8LBEE 
4002  SOTO  3900 
chained  prograa  for  jap 


100  ’ . geographic  locations  of  epos  — . 

105  SCREEN  2:HIDTH  30:0  OFF :CLS: CLOSE iDEFINT  A -Z 

110  DEF  SES:&H6800:BLOAD'usa.dat',0 

115  LOCATE  2,30:PRINT’Ar»y  CPO’s  in  CONUS' 

120  blink  the  dots  — 

125  FOR  ZZZ-1  TO  2 
130  FOR  PQS  - 1  TO  92 

135  READ  Xl.Yl 

140  PSET(X2,Y1),1:  PSET  (XUl.mi),!;  PSET(X1*1,Y1),1:PSET(X1,Y1*1),1 

145  NEXT 

150  RESTORE 

155  FOR  PQS: l  TO  92 

160  READ  XI, Y1 

165  PSET(X1,Y1) ,0:  PSET  (X1+1.Y1H) ,0:  PSET ( XI +1 , Y1 ) , 0 : PSET (XI , Y 1+ 1 1 ,0 
170  NEXT 
175  RESTORE 
ISO  NEXT  111 

132  DATA  480,105,550,52,400,105,440,40,375,65,490,75,30,20,510,75,435,90 
134  DATA  130,70,300,140,513,58,320,95,535,40,533,32,340,72,550,53,375,120 

136  DATA  560, 43, 500, 103, 402, 57, 300,30, 390, 110, 22, 85, 203, 77, 485, 105, 32, *2 

133  DATA  435,30,475,96,30,70,213,30,500,55,330,100,510,35,345,35,50,30 
190  DATA  545,46,140,70,340,110,510,95,15,75,440,42,570,40,395,128,500,60 

192  DATA  80, 27, 530, 70, 495,55, 415, 92, 430, 100, 370, 80, I4S, 112, 360.105, 30»2S 
194  DATA  525,78,300,100,535,83,490,110,215,82,100,108,365,100,430,115 
196  DATA  445,75,382,60,315,73,530,41,465,30,340,42,551,56,510,45,520,72 

193  DATA  525,58,425,34,402,60,427,72,533,38,300,120,535,55,425,63,415,82 
200  DATA  24,90,440,110,530,43,435,120,40,65,552,53,520,44,15,73,529,53 
202  DATA  340,125,290.110,230,130 

215  LOCATE  13,5:?RINT  '133  Operating  CPO’s' 

220  IF  INKEYJ:"  THEN  220 

225  LOCATE  20,5:FRINT  'plus  43  Overseas' 

239  IF  INXEYJ:"  THEN  230 

232  LOCATE  22,5:PRINT  'plus  40  Staff  CPO’s' 

233  IF  INKEYJ:"  THEN  233 

235  LINE  (250, 165)“ ! 370 , 177) , ,B 

236  LOCATE  22,35:?RINT  ’216  Total' 

249  [F  INKEYJ:"  THEN  249 

250  CHAIN  'BRIEF513', 2000, DELETE  100-250 
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SEVBBIN  JOHNSON’S  BBPOBT 


My  detail  to  assist  Mr.  Loren  Martindale  began  on  1  April  1985 
and  initially  consisted  of  telephoning  CPO’s  in  Army  and  later  in 
contacting  those  that  had  been  identified  as  "experts”  in 
classification.  When  this  portion  off  the  assignment  was  nearing 
completion  I  asked  and  Loren  agreed  that  I  could  tackle  the 
development  of  a  classification  "expert  system".  My  familiarity 
with  any  type  of  computer  system  was  limited  to  the  information 
needed  to  classify  ADP  jobs  and  a  couple  of  hours  running  a 
tutorial  on  an  IBM  PC  the  day  I  was  told  I  would  be  detailed  to 
assist  Loren,  29  March  1985.  My  personal  experience  included 
about  ten  years  as  a  classifier,  three  years  as  a  generalist 
( R&P , MER, PM&C ) ,  and  two  years  as  a  CPO.  So  it  appeared  to  Loren 
and  I  that  this  may  be  a  good  test  of  whether  it  would  be 
possible  for  reasonably  intelligent  personnel ists  to  develop  the 
skills  to  automate  their  own  functions. 

The  approach  taken  was  for  Loren  to  provide  me  an  hour  or  so  of 
hands  on  instruction  with  the  "Expert  Ease"  software  and  the  IBM 
PC  hardware  I  had  access  to  at  Oakland  Army  Base.  I  then  set 
about  reading  the  narrative  material  of  the  software  to  better 
understand  what  the  software  was  capable  of.  The  first  standard 
I  chose  to  develop  an  expert  system  on  was  the  Evaluation  Guide 
for  Typists.  The  only  problem  I  encountered  was  an  inability  to 
make  the  software  do  what  it  promised  because  I  was  applying 
typewriter  logic  to  the  keyboard.  Once  I  understood  better  the 
functioning  of  the  keys  all  went  quite  well  and  the  system  was 
developed  in  about  two  days.  It  was  tested  later  and  appears  to 
be  usable  by  operating  officials. 

With  my  appetite  whetted  with  that  experience  I  felt  ready  to 
tackle  something  with  a  little  more  complexity.  The  second 
standard  I  chose  was  the  Supervisory  Grade  Evaluation  Guide.  I 
felt  I  knew  that  standard  as  well  as  any  and  it  had  more  broad 
based  usage  potential  than  some  of  the  more  numerous 
transportation  family  jobs  that  I  was  most  familiar  with  at 
Oakland  Army  Base. 


Over  the  course  of  the  next  three  weeks  I  spent  a  number  of  hours 
on  the  computer  and  at  my  desk  working  out  a  logic  that  would 
allow  for  the  automation  of  that  standard.  The  process  with  the 
software  was  not  difficult  and  I  felt  that  the  results  were  quite 
complete  even  though  my  ADP  skills  have  remained  quite 
rudimentary. 

The  process  left  me  with  several  impressions  and  led  me  to 
several  conclusions.  The  automation  of  classification  standards 
does  not  require  an  expert  in  the  standard,  however  it  does  seem 
to  require  someone  who  has  a  strong  background  in  classification 
principles  and  policies.  I  would  venture  to  say  that  a  well 
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grounded  classifier  could  take  any  classification  standard  and 
automate  it.  The'  staffing  process  could  be  used  to  refine  the 
automation  and  build  in  occupation  unique  experience  from 
classifiers  who  have  had  extensive  field  experiences  in  the 
occupation.  Possibly  even  from  those  that  have  been  identified 
as  experts  in  this  project. 

The  use  of  expert  systems  if  not  used  totally  by  supervisors  and 
managers  has  a  good  chance  for  success  as  a  training  tool  for 
interns,  as  a  tool  to  help  classifiers  explain  classification  to 
supervisors  and  employees  and  finally  as  a  tool  to  speed  up  the 
classification  process.  Interns  could  more  quickly  develop  an 
understanding  of  a  classification  standard  by  being  able  to 
rapidly  see  how  different  sets  of  data  interact  in  order  to 
classify  a  position.  Access  to  the  building  blocks  of  an  expert 
system  could  also  help  to  better  understand  the  logic  upon  which 
the  standard  is  established.  The  explanation  of  classification 
could  be  enhanced  by  the  classifier  taking  a  supervisor  through 
the  automated  standard  and  thereby  being  able  to  point  out  areas 
of  specific  disagreement  and  faster  dialogue  on  what  caused  the 
selection  of  different  values  for  a  factor  in  disagreement.  Desk 
audits  are  not  the  most  efficient  means  of  gathering  data.  An 
expert  system  could  help  the  classifier  better  focus  on  the 
essential  data  necessary  for  classification  and  eliminate  or 
reduce  the  gathering  of  extraneous  data. 

I  also  tested  the  developed  SGEG  system  on  a  variety  of 
supervisory  positions.  The  test  was  run  with  second  and  third 
line  supervisors  and  as  can  be  seen  by  the  following  chart  there 
was  a  good  mix  of  Part  I  and  Part  II  jobs. 

Standard  Current  Class  Expert  Class  Differences 

Part  I  GS-2003-09  GS-2003-8  GS-9  based  on  non-supv  d. 

Part  II  GM-802-13  GM-802-14  Ovrstd  Mgr.  Aspects 

Part  I  GS-332-12  GS-332-12 

Part  I  GS-503-09  GS-503-08  Misclassif ication  ? 

Part  II  GS-2130-1 1  GS-2130-11 

Part  I!  GS-2130-12  GS-2130-12 

Part  II  GS-510-11  no  class  Based  on  non-supv  duties 

Part  II  GM-334-13  GM-334-13 

The  average  time  to  classify  these  positions  was  twenty  minutes. 
Although  the  second  time  through  the  same  part  for  a  different 
job  cut  down  the  time  almost  in  half. 

The  biggest  problem  in  the  test  was  that  the  language  was  more 
classifier  friendly  than  supervisor  friendly.  A  lot  of  the 
development  time  would  therefore  have  to  be  spent  in  formulating 
questions  that  are  supervisor  friendly.  The  classification 
standards  are  now  written  for  classifiers  and  need  substantial 
adaptation  in  order  to  be  consistently  understood  by  supervisors. 
For  instance  a  recurring  problem  surrounded  the  definition  of 
"manager".  For  many  of  the  supervisors  a  "manager"  was  no 
different  than  a  supervisor.  Some  of  these  limitations  may  have 
been  ironed  out  in  the  final  version  of  the  program  which  added 
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more  words  of  clarification  and  explanation. 

Finally,  the  extreme  number  of  factors  in  the  SGEG,  especially 
Part  I  proved  to  be  more  than  the  "Expert  Ease"  software  was 
capable  of  handling.  Therefore  the  final  version  of  Part  I 
should  accurately  classify  most  supervisory  jobs  but  the  physical 
limitations  of  this  software  will  cause  some  jobs  to  be 
misclassif ied.  More  powerful  software  or  a  different  logic 
approach  should  overcome  that  limitation.  The  nature  of  Part  II 
was  such  that  there  were  no  such  size  limitations  and  Part  II 
should  more  consistently  provide  correct  classifications  as  it 
currently  stands.  I  will  add  that  there  may  be  some  classifiers 
that  would  argue  some  of  the  logic  employed  in  the  standard  as  it 
currently  stands.  I  am  not  saying  it  is  perfect  in  every  way! 

The  three  weeks  development  time  appears  to  be  a  reasonable 
length  of  time  given  the  limited  system  expertise  I  had  when 
starting  the  project.  Today  a  similar  standard  would  take  less 
development  time. 

Although  no  attempt  was  made,  it  is  reasonably  clear  that  the 
expert  system  concept  could  easily  be  used  in  areas  other  than 
PM&C.  X-118  determinations  could  easily  be  made.  Crediting 
plans  could  be  easily  automated.  There  are  many  uses  in 
Technical  Services  especially  when  advising  employees  on 
retirement  options,  etc.  In  MER  disciplinary  actions  are  ripe 
for  the  use  of  expert  systems.  In  summary  any  personnel  function 
that  can  be  explained  to  someone  else  can  be  put  on  an  expert 
system.  Hopefully,  that  should  mean  all  areas  of  the  personnel 
office.  Inciden-tal  ly  it  is  feasible  for  individual  personnel 
offices  to  start  using  expert  systems  technology  right  now.  The 
software  in  available,  the  skills  are  easily  learned,  and  the 
personnel  expertise  resides  in  all  personnel  offices  to  one 
extent  or  another. 
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CRITICAL  ISSUES  QUESTIONNAIRE 


The  following  questionnaire  questions  have  been  posed  to 
approximately  70  professional  personnel ists .  Responses  were 
received  from  six,  and  their  comments  summarized.  T1  final 
pages  of  this  appendix  includes  the  reports  from  three  work 
groups  who  discussed  the  ideas  presented  at  a  professional 
development  seminar  held  in  the  Pentagon  on  2  May  1985. 

MIMIC  CURRENT  SYSTEM? 

To  what  degree  do  you  believe  an  expert  system  should  mimic  our 
current  system  with  respect  to  policies,  rules  and  regulations? 

Exactly/totally.  The  system  must  produce  results,  advice, 
decisions  and  actions  that  are  identical  to  those  that  are 
produced  by  personnel  people,  doing  their  Jobs  correctly,  under 
our  current  system. 

God  forbid!  The  current  system  is  pretty  s....  and  there’s  no 
reason  to  emulate  that. 

Close  resemblance.  The  system  must  closely  follow  general 
policies,  rules  and  regs,  but  some  of  the  specific  minute  details 
which  do  not  contradict  these  policies  could  be  handled  in  any 
way  most  amenable  to  efficient  operation  of  the  #  system. 
[EXAMPLES :  We  currently  utilize  large  numbers  of  forms  for  record 
purposes.  As  long  as  information  is  available,  is  it  crucial 
that  these  forms  continue  to  be  the  only  accepted  documents?  We 
currently  have  the  capacity  to  respond  to  ANY  question  posed  from 
staff  levels  above  us,  frequently  devoting  much  time  and  effort 
to  research  to  generate  non-recurring,  special  purpose 
information.  is  it  crucial  that  this  ability  to  ask  for  never- 
done-before  responses  from  operating  CPO’s  be  preserved? ] 

Checked  4  times. 

I  believe  the  Federal  personnel  system  needs  a  well-defined 
structure  that  provides  uniformity  in  the  way  employees  are 
treated.  Without  it,  we  would  have  a  horrendous  problem  with 
credibility.  As  it  is,  there  is  certainly  enough  discretion  and 
flexibility,  not  to  mention  latitude  in  interpretation,  in  the 
current  system  to  create  a  certain  amount  of  incredibility  in  the 
minds  of  employees.  For  example,  noncompetitive  actions  such  as 
reassignment,  or  reinstatement  to  a  grade  no  higher  than  last 
held,  are  widely  regarded  as  violations  of  the  merit  principle. 
Differences  in  the  application  of  the  highest  previous  rate  rule, 
a  discretionary  matter,  also  causes  consternation.  I  think 
people  expect  uniformity  and  anything  looser  than  what  we  now 
have  could  be  chaotic. 

Somewhere  between  this  one  and  the  next  one.  Ultimately  the 
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system  could  mimic  everything  we  desire.  At  first,  however,  it 
should  mimic  the  easy  stuff  from  a  regulatory  base. 

Close  resemblance  would  be  adequate  in  my  judgment.  Anything 
more  radical  would  be  difficult  to  get  approved,  i.e.  the  Civil 
Service  Reform  couldn't  change  the  rule  of  three. 

Loose  resemblance.  The  system  can  be  designed  to  follow  general 
principles ,  but  changes  to  current  regulations  and  rules  could  be 
incorporated  as  the  system  is  built.  [EXAMPLES:  The  merit 
promotion  principle  must  be  maintained,  as  well  as  the  principle 
of  equal  pay  for  equal  work.  However,  is  it  crucial  that  all 
current  regulatory  guidance  governing  the  merit  principle  be 
mimiced  by  the  system?  Is  it  crucial  that  jobs  be  classified  in 
strict  accordance  with  currently  published  OPM  standards? ] 

Checked  2  times. 

Although  a  functional  analysis  of  personnel  Mgt  might  conclude 
that  a  total  revamp  is  appropriate,  obtaining  the  necessary 
waivers  and  approvals  might  cause  the  expert  system  to  get  stuck 
in  the  bureaucracy  and  we’d  never  get  it  implemented.  We  don’t 
need  a  100*  mimic;  perhaps  meet  the  "intent"  rather  than  the 
letter  of  the  regulations. 

Total  revamp.  The  system  can  be  designed  to  create  an  entirely 
new  personnel  administration  and  management  environment  in  which 
even  basic  principles  are  changed.  [EXAMPLES:  The  principle  of 
equal  pay  for  equal  work  could  be  replaced  with  a  seniority 
system,  or  something  even  more  radically  different  such  as 
management  discretion .  The  principle  of  competition  for 
initial  competitive  appointments  by  certification  from  OPM  could 
be  replaced  by  new  rules  not  requiring  referrals  from  OPM.] 

Not  checked  at  all. 

SYSTEM  CHEATERS 

How  important  is  it  that  a  system  provide  continuous  oversight  to 
personnel  administration  and  management  as  it  is  being  performed 
by  managers  and  supervisors? 

Absolutely  essential.  Any  expert  system  must  also  provide  a 
means  to  prevent  any  deliberate  or  errorful  activities  on  the 
part  of  supervisors ,  managers  and  employees .  [EXAMPLE :  As  the 
system  classifies  a  job  based  on  supervisor  input,  it 
simultaneously  checks  for  overlap  of  functions ,  good  position 
management  principles ,  etc.,  and  prevents  the  supervisor  from 
cheating.  Additionally,  as  an  employee  performs  his/her  job, 
reports  of  tasks  performed  are  entered  by  the  employee ,  and 
inconsistencies  with  job  classification  are  reported  to  higher 
levels,  or  the  system  automatically  reclassifies  the  job  based  on 
tasks  actually  performed . ] 

Checked  4  times. 
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Sorry,  but  ay  general  view  is  that  it's  human  nature  to  try  to 
beat  the  system.  The  reason  we  have  a  vast  body  of  statutes  and 
regulations  now  is  that  the  Federal  Civil  Service  was  o.ice  a 
corrupt  spoils  system.  It  could  be  again.  Who  cares  about 
accountability  to  an  inanimate  system? 

I  think  the  problem  here  is  how  much  extra  would  it  cost  in  terms 
of  resources  to  provide  complete  oversight  to  ensure  that  no 
cheaters  get  by.  I  think  it's  important  but  people  can  note 
problems  on  a  "by  exception”  basis  to  catch  cheating. 

I  believe  oversight  is  essential  .:ot  so  much  because  of  cheaters 
but  because  it  takes  close  attention  &  effort  by  experts  to 
provide  continuous  maintenance  to  keep  an  organization  healthy  & 
vital. 

Yes,  unfortunately,  still  accountability  alone  is  not  enough,  for 
one  reason  we've  never  really  disciplined  a  cheater.  When  a 
supervisor's  valid  best  interests  are  to  cheat  then  he  or  she 
must  cheat  some  just  to  get  the  job  done. 

I’m  not  comfortable  with  either  extreme.  The  police  role  is,  I 
think,  incompatible  with  the  operation  of  an  expert  system. 
However,  neither  can  I  say  the  sky’s  the  limit  with  no  checks  and 
balances  unless  we  are  also  prepared  to  say  that  pay  accuracy, 
etc. ,  are  no  longer  objectives  of  the  Federal  Personnel 
management  system.  I  think  that's  the  real  question  here  -  what 
are  the  objectives  of  the  personnel  management  system?  If  the 
factors  you  describe  above  are  legitimate  objectives,  then  there 
must  be  some  controls.  If  not,  then  controls  are(  not  necessary. 

Not  important .  Supervisors  and  managers  are  responsible  for 
personnel  management  and  administration .  An  attempt  to  produce 
an  expert  system  that  plays  policeman  will  continue  to  duck  the 
issue  of  accountability. 

Checked  1  time. 

I’m  struggling  with  this  question,  but  I  definitely  lean  to  this 
answer.  Build  on  the  changes  in  manpower  "ceilings"  (the  FY85 
test)  -  and  see  how  far  we  can  get  management  accountable  for 
$$(something  tangible).  This  approach  implies  that 
classification  may  have  to  be  developed  at  a  later  stage  (when 
the  manpower  issues  straighten  out)  but  the  emphasis  on  supv 
assigning  duties  is  critical  and  should  not  be  mislaid. 

IMPERSONAL  PERSONNEL 

The  personnel  business  would  likely  become  very  impersonal  from 
an  operational  point  of  view  under  an  expert  system.  Thus,  a 
personnel  office  would  not  exist  that  would  provide  ombudsman 
services .  While  technical  advice  would  be  available  on  line, 
personal  counseling  services  in  light  of  this  technical  advice 
would  have  to  be  performed  by  supervisors .  What  are  your 
reactions  to  this? 


Horrified.  The  personal  advisory  services  provided  by 
personnel ists  is  one  of  the  most  important  functions  we  perform . 
The  Army  would  be  making  a  grave  mistake  to  give  up  the 
unmeasurable  benefits  that  result  from  the  many  specialists  that 
are  available  to  supervisors  and  employees  in  personnel  offices. 
Our  beneficial  effects  on  helping  to  maintain  good  morale  in  the 
workplace  by  being  there  must  be  preserved . 

Checked  2  times. 

I  believe  personal  advisory  service  by  personnelists  is  most 
essential  and  a  great  assistance  to  supervisors. 

I’m  not  horrified,  but  I  don’t  think  the  next  level  is 
sufficient.  Some  reduction  in  CPO  staffing  may  occur,  but  not  50% 
or  more. 

My  experience  with  personnel  offices  is  that  they  are  already 
impersonal  and  not  particularly  professional  so  no  loss  here.  I 
think  personnelists  are  responsible  for  a  large  amount  of  poor 
morale  due  to  their  lousy  service. 

This  is  not  such  a  loss  -  how  much  "personal  advisory  service” 
do  we  do  now? 

Concerned.  While  we  provide  valuable  morale  enhancing  services , 
these  services  could  be  insured  by  maintaining  a  small  staff  of 
personnelists  whose  role  as  management-employee  relations 
generalists  would  be  available  to  employees  and  supervisors . 
While  the  system  may  produce  the  technical  nitty-gritty  service 
to  supervisors  and  employees ,  the  role  of  these  generalists  would 
ensure  a  counseling  service  as  an  escape  valve  for  those  who 
cannot  get  it  from  their  supervisors . 

Checked  3  times. 

Disagree  that  they  are  simply  "morale  enhancing"  services.  Even 
after  the  expert  system,  there’s  a  need  for  management  advice  and 
support  on  a  technical  level;  that  is,  "how  best  to  assign 
duties",  position  management  options,  options/methods  for  filling 
jobs.  Where  decisions  remain,  there  is  need  for  person  to  person 
management  assistance. 

I  agree  here.  A  small  professional  staff  could  do  a  better  job 
if  taught  to  provide  service  and  counseling. 

Again,  I  have  a  rather  cynical  view  of  the  ability  of  supervisors 
to  provide  effective  personal  counseling.  They  are  often  too 
close  to  the  problem  or  a  part  of  the  problem.  I  believe  there 
will  be  a  need  for  some  personal,  personnel  contact.  Also,  there 
are  numerous  special  programs  I  believe  will  defy  being  built 
into  an  expert  system  that  will  need  personal  guidance  to 
maintain  them.  Examples:  Handicapped  hiring,  rehabilitation  of 
compensation  recipients,  control  of  sick  leave  usage. 


That's  O.K.  Supervisors  have  the  responsibility  for  this 
function.  Social  services  should  not  be  a  function  of  the 
personnel  office. 

Disagree.  "Social  services"  should  be  a  function  of  the  CPO. 
Most  supervisors  don’t  know  or  understand  the  personnel  rules  and 
I'd  be  reluctant  to  give  up  personnel  expertise  altogether. 

COSTS  TO  IMPLEMENT 

It  is  likely  that  an  expert  system  will  cost  a  considerable  sum 
of  money  to  even  get  it  to  the  testing  phase  of  operation.  Costs 
will  be  driven  by  various  factors  such  as  how  closely  it  mimics 
our  current  system,  what  kind  of  hardware  is  needed,  whether  or 
not  software  development  is  done  in  house  or  by  contractors ,  how 
much  of  our  business  is  included  in  such  a  system,  etc.  How  much 
do  you  feel  the  Army  should  commit  to  such  an  undertaking? 

Drop  the  idea  now.  Our  functions  cannot,  or  should  not  be 
performed  by  an  expert  system  in  any  way. 

Checked  1  time. 

An  expert  type  system  is  more  feasible  in  some  functions  than 
others  such  as  classification.  However,  I  don’t  think  it  is 
worth  the  effort.  I  can  see  an  expert  system  as  a  research  tool 
in  the  legal  field,  labor  relations,  etc. 

Move  very  cautiously.  Invest  no  more  than  maybe  $500,000 
initially.  Do  what  can  be  -donb  for  this  amount,  regardless  of 
how  limited  it  might  be  in  terms  of  what  is  does.  Assess  what 
came  out  of  the  effort  in  terms  of  what  it  does,  and  the  promise 
it  holds. 

How  can  you  suggest  $500,000  and  hold  without  some  concise 
estimate  of  the  cost  of  getting  to  a  showable  product! 

Move  somewhat  boldly.  Commit  to  a  test  (demonstration)  system  of 
considerable  scope  in  terms  of  the  functions  it  performs  (for 
example  classification  and  staffing)  but  of  limited  impact  on  an 
installation  mission  (for  example  one  or  two  organizational 
functions  at  an  activity) .  Place  it  in  an  operational  setting 
and  examine/evaluate  its  f easib Hi  ty /problems/potent ial .  The 
dollar  costs  can  be  whatever  they  are  to  do  this. 

Checked  5  times. 

Commit  to  a  test  (or  demo)  system  based  on  hard  cost  estimate. 
If  necessary,  fight  for  additional  funds  to  get  there.  Be 
flexible  about  what  functions  to  test  -  class  only?  staffing 
only?  Start  "small  scope"  so  we  can  study  impacts,  costs  & 
validity. 

If  there  is  a  positive  outcome  of  the  initial  developmental 
phase,  it's  a  waste  of  time  and  money  to  do  less  than  an 
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operational  test. 

I  feel  this  is  the  preferred  move.  We  should  go  for  this  level 
of  support. 

This  is  fairly  realistic  if  you  can  keep  it  under  $1  million.  We 
should  develop  a  PDIP  for  this  project  ASAP. 

Move  very  boldly.  Commit  to  a  significant  demonstration  project 
either  in  terms  of  scope  (i.e.  the  functions  to  be  included)  or 
impact  (i.e.  a  test  installation  at  which  all  activities  are 
serviced  by  the  system),  or  both.  Cost  should  be  no  object. 

I’d  like  to  see  this  happen,  but  having  watched  how  the  Army 
staff  operates  it’s  unlikely. 

IMPACT  ON  CAREERISTS 

A  fundamental  premise  of  creating  an  expert  system  is  it  will 
actually  save  money  by  replacing  personnel .  The  validity  of  this 
premise  is  currently  under  investigation,  and  may  not  be  proven 
or  disproves  until  an  actual  demonstration  project  is  undertaken . 
However,  if  an  expert  system  is  eventually  put  into  operation ,  it 
must  cost  less  to  provide  personnel  services  utilizing  it  than 
would  be  expended  under  a  system  where  people  do  the  work,  or 
there  is  very  little  reason  to  do  it.  How  do  you  feel  about  the 
idea  of  undertaking  a  project  that  has  as  its  basic  tenant,  the 
goal  of  replacing  occupational  opportunities  for  a  group  of 
people? 

Against  it.  It  matters  not  whether  it  can  be  done,  or  whether 
quality  or  manner  of  operation  are  changed.  It  is  simply 
undesirable  to  set  out  to  dismantle  an  occupational  field 
providing  employment  to  people. 

Checked  1  time. 

Currently  the  United  States  is  steadily  losing  its  middle  class 
due  to  foreign  competition  &  .other  reasons  which  is  most 
undesirable.  I  oppose  this  kind  of  thing  as  a  social  policy. 

For  it.  If  it  can  be  done,  it  should  be.  The  people  can  be 
retrained .  Doing  the  Job  at  less  cost  is  more  important  than  the 
impact  it  might  have  on  the  current  and  future  people  doing  this 
work. 

Checked  5  times. 

Some  "expertise"  will  still  be  required: 

a.  Maintain  the  expert  system. 

b.  Provide  person  to  person  contact  (generalist)  when 
necessary  for  decision-support. 

Also,  the  issue  is  not  just  "less  cost"  but 

1.  consistency 

2.  timeliness 
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non- 


3.  That  is,  better 

We’ll  carve  out  a  role  for  generalists  in  doing 

administrative  non-clerical  things. 

My  sense  is  that  it  will  be  a  very  gradual  process  for 

supervisors  to  become  adept  in  using  an  expert  system,  and  a 
considerable  amount  of  attrition  can  take  place  during  the  phase- 
in  period.  Further,,  it  seems  to  me  that  although  the  personnel 
staff  will  be  considerably  smaller,  those  who  remain  will  be 
doing  the  work  that  represents  the  best  part  of  personnel  work. 
The  drudgery  work  will  be  done  by  the  system. 

We  should  not  live  in  the  past,  new  roles  will  pop  out  as 
essential,  human  effort  will  be  raised  to  a  higher  plane. 

• 

Great  idea.  Personnelists  are  a  high  priced  group  of  paper 
shufflers.  It’s  unlikely  we’ll  ever  be  rid  of  them  all  but  maybe 
this  system  will  contribute  to  them  doing  a  better  job. 

HOW  TO  DEVELOP 

If  Army  undertakes  further  examination  of  using  expert  systems  in 
personnel  administration ,  it  would  be  wise  to  develop  a 
demonstration  (test)  of  such  a  system.  The  manner  in  which  this 
should  be  done  is  being  assessed  now.  If  a  demonstration  project 
were  undertaken ,  how  do  you  think  it  should  be  developed?  The 
following  statements  are  provided  to  stimulate  thoughts  in  this 
regard.  You  need  not  1  imit .  yourself  to  these  statements  only. 
Any  suggestions  you  may  have  are  welcome. 

I  believe  the  idea  should  be  dropped  at  this  time.  Do  not  invest 
any  more  effort,  time  or  money  in  the  expert  system  idea. 

It  is  important  to  proceed  very  cautiously,  perhaps  hire  a 
professional  expert  system  development  consultant  to  assess  the 
problem  before  undertaking  any  commitment  to  a  demo  project. 

The  demonstration  project  should  be  undertaken  very  soon  if  it  is 
at  all  technically  feasible. 

Set  up  a  special  group  in  D.C.  to  develop  it.  Then  select  an 
installation  at  which  to  test  it. 

Assign  the  project  to  a  major  command  to  develop  it  and  test  it. 
Army  to  fund  it. 

Army  to  assign  project  to  an  installation  directly.  Army  to  fund 
i  t . 

Keep  all  software  development  done  in  house. 

Contract  software  development . 

Any  project  office  must  have  direct  access  to  decision/policy 
makers,  but  it  is  very  important  to  undertake  even  the  initial 
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development  of  such  a  system  at  a  test  site  rather  than  at  a 
staff  level. 


Please  include  any  other  ideas  you  may  have  on  hot*  we  might 
proceed  to  develop  a  demonstration  project  if  one  seems  to  be 
feasible . 

The  originator  of  the  idea  should  continue  with  the  development 
of  a  demonstration  project  and  should  be  given  the  latitude  to 
select  the  help  he  needs  as  well  as  the  test  installation.  The 
reason  for  this  is  that  there  will  almost  certainly  be 
considerable  resistance  to  this  idea  among  personnelists  all  the 
way  up  to  top  staff  levels. 

To  have  a  chance  of  success,  it  must  be  Army  sponsored  and 
funded.  To  put  it  in  DC  would  likely  not  put  it  where  best 
access  to  technical  resources  may  be  located,  and  runs  a  high 
probability  of  "too  many  cooks  spoil  the  broth"  results.  To  put 
it  at  an  installation  and  under  their  jurisdiction  will  likely 
bump  up  against  too  many  people  who  see  their  way  of  life 
threatened.  Establish  a  task  force,  put  them  where  they  have 
access  to  necessary  technical  resources  to  develop  the  project, 
then  put  it  at  an  installation  to  test  it  with  oversight  and 
liaison  by  members  of  the  task  force.  The  task  force  must  have 
access  to  and  support  of  decision  makers.  But  it  must  also  have 
a  lot  of  freedom  to  do  this  job. 

Continue  to  develop  and  test  blue  collar- jobs,  classification  and 
demonstrate  program  at  a  test  site.  Depending  on  resources, 
expand  to  low  level  white  collar  jobs  and  other  functions. 

If  you  proceed  with  a  test,  demonstration  project  -  you  should 
take  a  small  bite  of  the  most  feasable  area  such  as 
classification.  I  don’t  think  it  is  very  feasable  in  many 
functions  such  as  staffing,  employee  relations. 

Very  critical: 

1.  Central  funding  and  leadership 

2.  Operating-level  expertise 

3.  Ownership:  keep  it  moving 


ATTENTION  OF 

PECC-CID 


DEPARTMENT  OF  THE  ARMY 
U.S  ARMY  CIVILIAN  PERSONNEL  CENTER 
200  STOVALL  STREET 
ALEXANDRIA.  VA  22332  0300 


17  MAY  19cj 


MEMORANDUM  THRU  Chief,  HQ  Civilian  Personnel  Office 
U.  S.  Army  Materiel  Command 

FOR  Director  Civilian  Personnel 

SUBJECT:  Nick  Hoge  Professional  Development  Seminar  -  Group  II  Discussion 
Report 


1.  Mr.  Martlndale's  presentation  and  essay  "A  case  for  Expert  Systems  in 
Personnel  Administration"  set  the  tone  for  enthusiastic  and  thought  provoking 
discussion  about  the  use  of  automation  In  the  every  day  business  of  civilian 
personnel  management.  This  report  covers  the  primary  areas  of  discussion  and 
recommendations  made  during  the  session. 

2.  General  Discussion. 

The  group  viewed  the  advance  of  automation  In  the  personnel  business 
Including  expert  system  application,  to  be  a  positive  step  forward.  Some 
concern  was  expressed  that  as  more  and  more  of  the  personnel  functions  become 
automated  we  run  the  risk  of  losing  the  personal  interaction  that  goes  on 
today.  Many  of  the  group  participants  disagreed  stating  that  automation 
would  reduce  the  paper  log  Jams  and  in  reality  allow  more  time  for  the 
personal  Interaction  desired.  The  group  looked  upon  automation  not  as  a 
threat  but  rather  a  support  tool. 

3.  Discussion  of  the  "Ideas  for  Consideration". 


a.  Is  the  proposal  vail d/of  value  to  the  civilian  personnel  community 
within  Army? 

The  group  consensus  was  a  resounding  "yes",  but  a  lot  of  work  needs  to  be  done 
to  have  It  readily  accepted  within  the  personnel  and  serviced  communities. 

This  work  Includes  training,  organizing,  and  funding  support  which  are 
discussed  more  fully  In  the  specific  points  that  follow. 

b.  Should  the  proposal  be  expanded  or  contracted? 

The  proposal  should  be  expanded  to  the  other  personnel  programs.  The 
recommendation  was  made  to  develop  applications  for  selected  programs  and 
distribute  them  among  Interested  CPOs  for  testing  and  evaluation. 

Recognizing  that  this  approach  Is  not  as  simplistic  as  It  sounds,  the  group 
suggested  that  Army  first  develop  and  refine  one  program  for  Army-wide  use. 
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PECC-CIO 

SUBJECT:  Nick  Hoge  Professional  Development  Seminar  -  Group  II  Discussion 
Report 

c.  What  are  the  major  obstacles  to  Implementing  such  a  proposal? 

Major  obstacles  identified  were: 

1.  Getting  the  funding; 

2.  Overcoming  perceived  lack  of  commitment  and  resistance  to  change; 

3.  Identifying  and  maintaining  the  human  expert  base. 

d.  Should  such  a  proposal  be  Implemented?-  How? 

The  group  opinion  was  that  Indeed  such  a  proposal  should  be  Implemented  using 
the  formal  Project  Management  approach.  This  approach  would  provide  the 
level  of  control,  visibility  and  relative  stability  needed  to  support  funding 
and  manpower  Issues. 

e.  What  about  the  costs  of  the  system?  People?  Money?  Equipment? 

Time? 

The  group  agreed  that  these  Issues  couldn't  be  fully  discussed  and  answered 
In  the  session  because  of  the  variables  that  exist.  The  group  recognized 
that  these  Issues  must  be  carefully  considered,  researched,  and  resolved 
before  automated  application  could  be  developed.  These  Issues  would  be 
addressed  in  the  Mission  Elements  Needs  Statement  for  the  project.  * 

f.  What  are  the  Impacts  on  the  Civilian  Personnel  field?  Philosophical? 
Managerial?  Accountability? 

There  are  numerous  pro  and  con  Impacts  In  Introducing  expert  system  concepts 
to  the  personnel  business.  Philosophically,  we  need  to  consider  how  we  will 
retain  and  maintain  the  small  human  expert  knowledge  base.  Once  expert 
systems  were  In  place,  the  core  knowledge  base  required  at  each  site  would  be 
reduced.  We  would  then  have  to  rely  on  a  smaller  (elite?)  base  to  maintain 
the  systems  and  Incorporate  policy  and  legislative  changes.  Also,  in  the 
philosophical  vein  there  are  certain  cultural  barriers  that  must  be  worn 
down.  There  still  exists  the  reluctance  to  change  and  the  reluctance  to 
trust  machines  to  make  Judgmental  decisions.  Which  leads  to  the  managerial 
Impacts.  Management  must  be  willing  and  committed  to  accelerating  the 
automation  program  for  civilian  personnel  In  Army,  stepping  up  training  of 
personnellsts  and  managers  and  establishing  sound  plans  to  do  all  of  that. 

From  the  standpoint  of  accountability  -  with  respect  to  the  PM&C  application 
presented  -  the  group  agreed  that  accountability  still  rests  with  the 
supervisor.  This  concept  must  be  fostered  If  such  an  expert  system  Is  to 
succeed.  Along  those  lines  the  group  discussed  whether  or  not  expert  systems 
or  automation  in  general  would  thwart  would  be  "system  cheaters".  The  group 
agreed  that  automation  would  not.  The  answer  to  dealing  with  "system 
cheaters"  Is  not  automation  but  rather  training  and  other  means  to  instill 
ethical  management  practices. 
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PECC-CIO 

SU8JECT:  Nick  Hoge  Professional  Development  Seminar  -  Group  II  Discussion 
Report 

g.  How  does  this  system  interrelate  to  other  automation  initiatives? 
VIA8LE?  ACPERS?  Office  automation?  There  is  a  definite  Interrelationship 
between  expert  system  application  and  current  and  future  automation 
Initiatives.  That  ts  why  the  group  proposed  the  formal  project  management 
approach  to  developing  expert  systems  applications.  It  Is  Imperative  Army 
have  a  grasp  of  Its  automation  programs  so  automation  dollars  can  be  more 
wisely  spent. 

% 

4.  It  was  a  pleasure  to  be  a  part  of  the  discussion  group  and  to  see 
personnellsts  actively  Interested  and  so  well  versed  in  today's  automation 
Issues. 


Leslie  D.  Smoot 

Personnel  Management  Specialist 
ACPERS  Project  Office 
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ANCIV-XTM 

MEMORANDUM  FOR  DIRECTOR  OF  CIVILIAN  PERSONNEL 


8  May  1985 


SUBJECT:  Nick  Hoge  Award  Professional  Development  Seminar— INFORMATION 
MEMORANDUM 


1.  Purpose.  To  furnish  information  on  Group  Ill's  discussion  concerning  the 
impact  and  application  of  expert  system  technology  on  civilian  personnel  mana¬ 
gement  and  administration.  The  information  emanates  from  the  suggested  "Ideas 

for  Consideration"  handout  provided  to  panel  members  (enclosure  1).  A  list  of  panel 
members  is  at  enclosure  2. 

2.  Information. 

a.  Is  the  proposal  valid/of  value  to  the  civilian  personnel  community 
within  Army?  Yes. 

(1)  Civilian  personnel  knowledge  would  be  better  catalogued  and  more 
easily  retrievable. 

(2)  The  system  could  be  used  to  validate  standards  as  it  is  programmed 
and  used. 

(3)  The  computer  interfaces  with  nearly  all  programs  in  today’s 
environment . 

(4)  Consistency,  standardization  and  speed  would  be  improved. 

(5)  The  advantage  of  involving  the  supervisor  would  minimize 
classifier /supervisor /managerial  problems . 

(6)  It  would  give  the  personnelist  more  time  to  do  "expert"  work. 

(7)  DA  would  attain  a  better  informed  managerial  work  force. 

b.  Should  the  proposal  be  expanded  or  contracted?  Expanded. 

(1)  The  DCSPER  organization  could  attain  full  automation  in  10-15 
years,  i.e.  ACPERS,  DA  Office  Technology  Project,  and  Expert  Systems. 

(2)  Consideration  should  be  given  to  a  pilot  project  with 
expansion,  if  proven. 
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SUBJECT:  Nick  Hoge  Award  Professional  Development  Seminar— INFORMATION 
MEMORANDUM 

c.  What  are  the  major  obstacles  to  implementing  such  a  proposal? 

(1)  The  need  for  a  strong  quality  control  to  avoid  problems  with  the 
currency  of  the  data  base  is  critical.  How  many  safeguards  do  we  need? 

(2)  Supervisors /Manager s/Personnelist  would  all  require  PC/Micro 
equipment . 

(3)  An  intensive  training  program  would  be  required  to  assure  current 
and  future  supervisors/managers/personnelists  are  trained  and  disciplined  in 
the  use  of  the  system. 

(4)  Not  everyone  agrees  with  the  "school  of  thought,"  consequently 
high  levels  of  resistance. 

(5)  There  is  a  concern  that  qualification  standards  are  not  vali¬ 
dated  and  we  would  be  making  bad  situations  worse. 

(6)  Some  functions  are  easier  to  automate  than  others. 

(7)  Idea  is  idealistic. .. supervisors  classifying  jobs.  Who  would 

control? 

(8)  There  was  a  concern  for  classifiers  validation;  interface  with 
interviews  and  classification  audit;  concern  for  loss  of  expert  base. 

(9)  The  expertise  in  the  ADP  community  on  expert  system  is  not  at  all 
that  great  at  this  time. 

(10)  Hardware  is  presently  unavailable;  the  need  for  mainframe  equip¬ 
ment  for  massive  storage  of  data  is  required. 

(11)  The  requirement  for  the  same  full-time  functional  experts  to  sup¬ 
port  development  of  the  life  cycle  for  the  system  (5-10  years)  is  not 
realistic. 

(12)  There  is  a  potential  cost  and  labor  management  relations  barrier 
against  the  project,  e.g.  AMC  depots  are  under  Army  Industrial  funds  and  have 
strong  unions. 

(13)  What  happens  to  grade  control? 

(14)  The  loss  of  personal  contact  with  serviced  activities. 

(15)  Presently,  not  enough  resources  for  ACPERS  let  alone  an  expert 

system. 
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SUBJECT:  Nick  Hoge  Award  Professional  Development  Seminar— INFORMATION 
MEMORANDUM 

d.  Should  such  a  proposal  be  implemented?  Yes.  How? 

(1)  Establish  a  plan  of  action,  milestones  to  include  quality  control 
and  training,  and  a  formal  project  officer. 

(2)  Consider  the  piece-meal  bases,  i.e.  one  function  versus  all  at  one 
time  approach. 

(3)  Use  of  mainframes,  VIABLE  sites,  DA  Office  Technology  Project, 
supplemented  with  additional  software,  and  communication  technology,  should  be 
considered  for  on-line  system. 

(A)  Purification  of  classification  standards  would  be  realized  as  new 
system  is  utilized.  Would  require  OPM  to  be  pro-active. 

(5)  Set-up  test  sites  (pilot  study),  establish  phase  plan  for 
classification  series;  expand  based  on  pilot  study;  and  delegate  different 
series  to  different  installations  with  a  task  group  control  to  assure  con¬ 
sistency. 

(6)  Give  consideration  to  China  Lake  concept  (demonstration  type 

project). 

e.  What  about  the  costs  of  such  a  system?  People?  Money?  Equipment? 

Time? 

(1)  Cost  and  resources  would  be  high  initially  but  long  range  benefit 

is  great. 

(2)  There  would  be  a  long,  long,  period  of  education. 

(3)  Cost  of  equipment  should  be  coordinated  with  ISC  for  planning, 
budget,  software,  and  hardware,  e.g.  VIABLE. 

(4)  Action  lends  itself  to  Learning  Centers;  consideration  should  be 
given  to  the  training  function  being  responsible. 

f.  What  are  the  impacts  on  the  Civilian  Personnel  field?  Philosophical? 
Managerial?  Account ablity? 

(1)  A  new  way  of  doing  business  would  have  to  be  realized. 

(2)  Would  require  committment  of  all  concerned. 

(3)  The  variety  of  socio-economic  needs  of  our  work  force  should  be 
considered. 
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SUBJECT:  Nick  Hoge  Award  Professional  Development  Seminar— INFORMATION 
MEMORANDUM 

(4)  We  should  not  assume  the  system  will  operate  in  a  vaccum;  devoid 
of  human  intervention. 

(5)  The  expert  level  of  personnel  people  could  realize  some  degrada¬ 
tion. 

(6)  How  many  safeguards  do  we  need? 

(7)  Degree  of  fear;  endangered  species;  organization  change. 

g.  How  does  this  system  interrelate  to  other  automation  initiatives? 
VIABLE?  ACPERS?  Office  automation? 

(1)  The  expert  system  would  be  utilized  by  all  DA  Civilian  Personnel 
Offices  and  therefore,  would  be  considered  a  Standard  Army  Management 
Information  Systems  (STAMIS)  and  have  VIABLE  constraints;  system  should  not  be 
maintained  by  individual  CPO's. 

(2)  DA  Office  Technology  Project  would  have  to  be  upgraded  to  sup¬ 
port  expert  systems. 

(3)  ACPERS  is  a  data  base  management  system  and  presently,  expert 
systems  software  does  not  intergrade. 

3.  The  subject  aroused  the  curiosity  and  interest  of  the  panel  members.  The 
agenda  for  the  seminar  seems  to  be  a  good  approach  and  should  work  well  in 
future  seminars. 


2  Ends 

1.  Ideas  for  Consideration 
Handout 

2.  List  of  Attendees 


Deputy  Civilian  Personnel  Director 


Gladys  M.  Tulloch/475-1601 
Typed  by  Pamela  J.  Ransom 


DEPARTMENT  OF  THE  ARMY 
WALTER  REED  ARMY  MEDICAL  CENTER 
WASHINGTON,  D  C.  20307  -5001 


SUBJECT:  Group  I  -  Discussion  Group  Report 


THRU: 


TO:  Mr.  Raymond  J.  Sumser 

Director  of  Army,  CPO 
PECC-CID  Room  3N69 
200  Stoval  Street 
Alexandria,  VA  22332-0300 


1.  The  purpose  of  this  report  is  to  provide  a  feedback  -  synopsis 
of  the  many  comments,  remarks,  and  most  importantly  the  enthusias¬ 
tic  discussion  generated  by  Mr.  Loren  D.  Martindale's  paper,  "The 
Case  For  Expert  Systems  For  Personnel  Administration." 

2.  Mr.  Martindale's  presentation  was  excellent.  It  exhibited  the 
viability  of  the  expert  system;  -  a  selling  tool;  an  educational 
tool;  a  working  tool;  to  assist  in  solving  personnel  problems. 

He  captured  everyone's  attention  during  his  introductory  course 
analysis  remark  regarding  "Eliminating  4500  personnel  specialist 
out  of  6200  careerist,"  currently  in  the  system.  However,  later 
discussion  revealed  many  personnel  specialist  may  switch  to  other 
fields  within  the  personnel  system  and  that  attrition  and  "phase  in 
time"  would  assist  in  accommodating  any  reduction  in  personnel. 

3.  The  major  obstacles  to  implementing  such  a  proposal  would  be  in 
our  opinion: 

a.  money  -  upfront  -  expenses,  acquisition,  installation, 
training,  and  maintenance; 

b.  training  of  supervisors  in  the  system  approach  to  personnel 
management ; 

c.  acceptance  by  managers  -  history  of  past  performance  with  a 
system  approach  may  pose  a  creditability  problem; 

d.  lack  of  personal  contact; 

e.  abuse  of  the  system; 

f.  who  will  be  accountable  for  the  sysL^m; 
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g.  the  system  will  demand  a  higher  level  more  intense  and  a 
broader  area  of  expertise  in  the  personnel  function(s)  by  the 
personnelist  than  that  which  is  now  possessed  by  most  personnelist ; 

h.  an  unknown  impact  factor:  personnel  information  is  a  con¬ 

cern  of  the  Information  System  Command,  (ISC),  Reference  -  ISC  letter 
subject:  Information  Management  Master  Planning  (IMMP)  Guidance, 

dtd  28  March  1985,  how  will  these  organizations  affect  the  personnel 
function? 

4.  The  consensus  of  the  group  regarding  the  implementation  of  the 
system  is  to  follow  a  non-combative  role.  For  example,  implement 
the  classification  function  later  and  introduce  a  ^helping  hand" 
service  approach  in  the  area  of  employee  benef its^jft  may  be  intro- 
ducted  by  phases : 

Phase  I  -  a.  Health  Insurance  selection  from  among  120  plans 
available  in  the  D.C.  area  -  based  upon  family 
health,  cost,  etc;  . 

b.  Life  Insurance  selection; 

c.  Retirement; 

d.  Training  and  Development. 

-  Phase  II  -  a.  Employee  Relations 

(1)  Local  -  problems  and  decisions 

(2)  MSPB  decisions 

b.  Labor  Relations 

(1)  Local  -  local  problems  and  decisions 

(2)  FLRB  decisions. 

(NOTE:  Several  private  sector  firms  have  computer  assistance  case 

decisions  systems  in  operation). 

-  Phase  III  -  Recruitment  and  Placement 

(NOTE:  FORCOM,  Huntsville,  Alabama  has  a  system  in  place  and 

operating) . 

-  Phase  IV  -  Position  Classification  and  Wage  Survey  -  (To  include 
total  compensation  package). 

5.  Is  Mr.  Martindale's  proposal  valid?  Is  it  of  value  to  the  Army 
Personnel  Community? 

-  The  Group  I  answers  to  these  two  questions  is  that  many  of  us  in 
the  personnel  field  are  already  moving  toward  this  direction. 

Perhaps  in  a  non-directional  non-centralized  approach.  For  example. 
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Walter  Reed  Army  Medical  Center  -  has  completed  a  computer  approach 
to  the  nurse  occupation,  610  series.  The  number  of  nurse  position 
descriptions  have  been  reduced  from  95  position  descriptions  to  23. 
Reduced  the  write-up  time  from  4  to  5  hr  to  approximately  45  minutes. 
Mr.  Mart indale's  presentation  has  sent  us  back  to  the  610,  621,  and 
620  series  to  refine  the  program  along  the  lines  of  Mr.  Mart  indale's 
proposal.  In  addition,  Walter  Reed  Army institute  of  Research  has 
approved  the  purchase,  installation‘7^2aKa training  of  personnelist  to 
computerize  all  WRAIR  personnel  systems.  Many  of  Mr.  Martindale's 
ideas  will  be  incorporated  in  this  system. 

-  Group  I's  opinion  is  that  the  proposal  is  valid  and  it  has  great 
value  to  the  Army  Personnel  Community.  Collectively,  the  group's 
opinion,  many  operating  civilian  personnel  offices  are  expanding  in 
this  area  and  there  should  be  centralized  guidance  and  expertise 
available  to  the  CPOs . 

6.  Implementation.  The  group  recommendations: 

a.  expand  the  proposal  to  do  a  feasibility  study  by  utilizing 
a  demonstration  project; 

(1)  establish  sub-demonstration  projects  at  several  different 
geographic  locations, 

(2)  each  sub-demonstration  project  will  work  one  functional 
area  of  personnel  or  a  smaller  component  of  it, 

(3)  to  ensure  a  conceputal  understanding  of  the  application 
of  the  entire  AI  to  the  total  personnel  system.  All 
personnel  functions  should  be  studied  simultaneously. 

(4)  for  control  and  comparative  analysis,  demonstration 
projects  should  be  established  where  there  is: 

-  considerable  collection  of  expertise,  equipment,  and 
application  and  operating  experience  with  AI  within 
the  CPO; 

-  less  than  what  is  discussed  in  4  (a)  above; 

-  no  expertise,  no  equipment,  and  no  computer  equipment. 

b.  It  is  further  recommended  that  Mr.  Martindale  be  placed  in 
the  position  of  an  operating  manager  for  the  demonstration  project. 


Civilian  Personnel  Office 
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CONTACTS  PROVIDING  INFORMATION/CONSULTATION 


Advanced  Information  &  Decision  Systems.  Mountain  View,  CA.  Mr. 
Cliff  Reid.  (415)  941-3912. 

Balistics  Research  Labs.  Aberdeen  Proving  Ground.  Mr.  Mort 
Hershburg.  AVN  298-6661. 

Soldiers  Support  Center.  Ft.  Ben  Harrison,  IN.  Ma  j .  Ken  Rose. 
AVN  699-3791. 

Office  of  Naval  Research.  Arlington,  VA.  Ms.  Susan  Chipman. 
(202)  696-4318. 

Arthur  Anderson  Co.  Phoenix,  AZ.  Mr.  Steven  Dick.  (602)  257- 
9234. 

Teknowledge.  Los  Angeles,  CA.  Mr.  Allen  Smith.  (805)  495-8265. 

Stanford  University.  Palo  Alto,  CA.  Mr.  John  McCarthy,  Mr.  Tom 
Rindsleisch,  Ms.  Sandy  Learner,  Mr.  Gordon  Bower. 

IntelliCorp.  Menlo  Park,  CA.  Mr.  Fred  Cummins.  (415)  853-5593. 

Lisp  Machine,  Inc.  Los  Angeles,  CA.  Mr.  Kenneth  Gruber.  (213) 
642-1116.  Santa  Clara,  CA.  Mr.  Warren  Goddard.  (408)  496-1151. 

Scientific  Systems,  Inc.  Cambridge,  MA.  Mr.  Webb  Stacy.  (617) 
661-6364. 

Software  A&E.  Arlington,  VA.  Mr.  Tony  Magliero.  (703)  276- 
7910. 

Smart  Systems  Technology.  McLean,  VA.  Mr.  Eamon  Barrett.  (703) 
448-8562. 

Inference  Corporation.  Los  Angeles  and  San  Francisco,  CA.  Mr. 
Pete  Larson,  Mr.  Shelton.  (213)  417-7997,  (415)  461-0513. 


And  the  many  professional  personnel ists  within  the  Department  of 


SCOPE  OF  WORK  STATEMENT 


The  contractor  will  provide  to  the  Department  of  Army  an  expert 
system  which  will  accomplish  all  of  the  tasks  specified  herein. 
The  system  will  operate  in  an  integrated  fashion,  mimic  the 
expertise  of  experts  provided  by  the  Army  as  models,  and  be  free 
of  errors,  in  both  programming,  and  expert  decisions  rendered  by 
the  system. 

GENERAL  DESCRIPTION.  The  system  will  perform  the  functions  of  a 
position  classification  specialist  in  an  operating  civilian 
personnel  office.  It  will  classify  jobs,  provide  position 
management  advice,  make  Fair  Labor  Standards  Act  determinations, 
advise  on  hazard  pay  and  environmental  differential  pay, 
determine  functional  classifications,  make  determinations  of 
supervisory  code,  produce  job  descriptions  and  evaluation 
statements,  maintain  records  of  its  use  (responses  from  its 
users),  and  work  in  a  mixed-initiative  mode.  The  system  will  be 
directly  usable  by  supervisors  and  employees  of  the  activity  from 
computer  terminals  located  at  several  locations  throughout  the 
installation . 

FOUNDATION  OF  THE  SYSTEM.  The  system  will  use  as  the  basis  for 
rules  the  current  Office  of  Personnel  Management,  Department  of 
Defense,  and  Department  of  Army  rules,  regulations,  manuals,  and 
policies  which  are  in  published  form.  Interpretation  of  these 
published  rules  will  be  provided  by  Army  to  the  contractor  as 
required. 

EXPERTS.  The  Department  of  Army  will  provide  human  experts  to 
the  contractor  for  use  in  the  development  of  the  system.  Experts 
will  serve  as  models  for  various  aspects  of  the  system 
development.  Experts  will  provide  to  the  contractor  the  knowledge 
possessed  of  interpretation  of  regulations,  as  well  as 
judgmental  knowledge  needed  for  decision  processes.  Additional 
experts  in  the  same  subject  matter  area  may  be  provided  by  DA  to 
test  the  system,  and  make  adjustments  to  the  decision  processes 
used.  All  differences  in  expertise  among  experts  will  be 
resolved  by  the  DA  project  manager. 

SYSTEM  PERFORMANCE.  Following  are  the  criteria  the  system  must 
perform. 

1.  The  system  will  be  able  to  link  to  and  use  Army  standard  data 
base  systems  such  as  ACPERS. 

2.  The  system  must  contain  the  ability  to  properly  classify  any 
and  all  jobs  in  the  organizational  entity  prescribed. 

3.  The  system  must  provide  this  classification  service  in  an 
interactive  query  format  available  to  supervisors  establishing 
new  jobs,  or  redefining  existing  jobs. 


4.  Classification  decisions  on  positions  rendered  by  the  system 
will  be  recorded  in  an  organizational  file  of  the  activity.  This 
organizational  file  will  contain  sufficient  information  on  all 
positions  for  use  by  any  other  part  of  the  system  for  operation. 

5.  The  system  will  maintain  an  audit  trail  of  the  classification 
decision  process  it  used  and  store  it  with  the  final  product  in 
the  organizational  file. 

6.  The  system  will  provide  security  so  that  only  authorized 
users  may  utilize  it  for  their  own  positions. 

7.  The  system  will  produce  a  job  description  similar  to  current 
Army  practices. 

8.  The  system  will  produce  an  evaluation  rationale  readable  by 
humans  that  justifies  the  classification  on  the  basis  of  criteria 
contained  in  OPM  classification  standards. 

9.  Position  management  advice  will  be  readily  available  to 
managers  that  takes  into  consideration  such  factors  as  average 
grades,  supervisory  ratios,  trends,  costs,  career  ladders,  grade 
controlling  duties  among  all  positions,  functional 
responsibilities  and  similar  matters. 

10.  Decisions  about  hazard  pay  and  environmental  differential 
pay  will  be  provided  by  the  system. 

11.  Supervisory  codes  will  be  provided  by  the  system. 

12.  The  system  will  make  accurate  determinations  regarding  the 
Fair  Labor  Standards  Act  for  all  positions  within  its  purview. 

13.  The  system  will  be  capable  of  handling  mixed  jobs. 

USER  INTERFACE.  Supervisors  are  the  primary  users  of  the  system. 
Thus,  the  system  must  be  designed  to  interact  with  these  who  are 
not  knowledgeable  of  personnel  systems,  but  are  knowledgeable  of 
the  jobs  under  their  supervision.  Supervisors  will  also  have 
varying  degrees  of  computer  literacy,  thus  the  system  must  be 
capable  of  eliciting  necessary  queries  to  users  in  consideration 
of  the  least  knowledgeable  computer  user.  The  system  must 
utilize  a  query  mode  that  is  thorough,  understandable  by  those 
who  use  it,  and  tolerant  of  mistakes,  unintentional  wrong  input, 
and  inconsistent  responses  by  users. 

SYSTEM  EXPANSION.  As  the  system  is  designed,  areas  of  future 
possible  expansion  will  be  identified  to  the  contractor,  so  that 
the  design  will  provide  a  ready  ability  to  accept  the  expansion. 
For  example,  new  jobs  will  be  created  in  the  future  for  which  the 
system  was  not  initially  designed  to  classify.  Expansion  to 
include  these  new  jobs  must  not  be  hindered  by  the  system 
provided  by  the  contractor.  Also,  new  functions  may  be  added  to 
the  system  in  the  future,  e.g.  employee  disciplinary  actions  and 
others.  The  system  must  not  be  designed  in  such  a  manner  as  to 
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hinder  or  prohibit  future  functional  expansion. 


The  contractor  will  provide  for  participation  of  government 
project  office  personnel  in  the  development  of  the  prototype  in 
order  that  these  personnel  become  proficient  at  understanding  the 
architecture  of  the  system,  its  inferencing  strategies,  and  its 
knowledge  base.  The  completed  system  must  be  maintainable  and 
expandable  by  government  personnel. 


EXECUTIVE  SUMMARY 


There  are  a  number  of  areas  in  which  expert  systems  technology 
can  be  applied  to  civilian  personnel  administration  functions. 

Potentially  the  most  beneficial  application  of  this  technology  is 
in  the  area  of  systems  that  can  be  used  directly  by  supervisors 
to  perform  personnel  functions. 

If  used  directly  by  supervisors/employees,  expert  systems  could 
replace  personnelists  who  currently  perform  the  work. 

POTENTIALLY,  several  million  dollars  per  year  could  be  saved  in 
salary  costs  if  expert  systems  were  designed  and  allowed  to 
perform  functions  of  personnel  office  employees. 

In  order  to  best  assess  the  potentials  of  expert  systems 
technology  in  personnel,  a  working  prototype  should  be  developed 
and  tested  in  an  operational  setting. 

The  position  management  and  classification  function  of  personnel 
offers  the  best  area  for  prototype  development. 

The  cost  of  prototype  development  for  this  R&D  effort  will  be  in 
the  neighborhood  of  $1.5M  to  $3.0M. 

It  is  recommended  an  R&D  project  office  be  set-up,  staffed, 
resourced  with  equipment,  and  charged  with  prototype  development. 

It  is  recommended  the  project  be  conducted  as  an  in-house  effort. 

It  is  estimated  a  prototype  could  be  built  in  two  to  three  years 
by  a  staff  of  7  people  (knowledge  workers  plus  support  staff). 

The  prototype  would  provide  all  PM&C  services  to  a  test 
organization  and  operate  at  the  expert  level  of  accuracy. 
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